Specialist-model research desk

Scorecard

Frequently asked questions

Frontier Development research deskSection 09 · Page 10 of 13Last updated 26 October 2026

Can we use this to publish a best-provider ranking?

Only after a separate, fair assessment with comparable evidence, scope and disclosures. Public platform descriptions establish an offering, not a completed supplier score.

Why does unknown contribute zero?

It makes the numerical total conservative while keeping missing evidence visible as U. Risk-management guidance supports documenting uncertainty, but this particular arithmetic is an editorial procurement choice.

Can we change the weights?

Yes: set them to the buyer’s priorities before reviewing proposals and keep the total at 100. The evaluation principles remain useful, but the weights are not prescribed by that source.

Does a score above 80 mean the model will work?

No: the score reflects the evidence and requirements assessed at that stage. Production quality still needs a protected test; group-aware splitting is one possible component of that evaluation design.

Should supplier payment influence the result?

No: record and disclose any commercial relationship, and keep it out of technical ratings. The central evaluation principle is to judge against the buyer’s task and evidence, consistent with a concrete error-led approach.

TRACE THE EVIDENCE

Sources cited on this page

  1. platform descriptions →www.together.ai
  2. Risk-management guidance →www.nist.gov
  3. evaluation principles →www.anthropic.com
  4. group-aware splitting →scikit-learn.org
  5. a concrete error-led approach →hamel.dev