Specialist-model research desk

Scorecard

How should we judge a specialist-model supplier?

Frontier Development research deskSection 09 · Page 1 of 13Last updated 26 October 2026

Judge the proposed plan against a common task and evidence standard before comparing prices. A high weighted score must not override a failed critical requirement or unresolved permission.

This is a desk-designed procurement rubric, not a validated ranking of providers. No named supplier has been scored in this pack. Agree the weights before seeing proposals and disclose commercial relationships before an evaluator scores them.

TRACE THE EVIDENCE

Section source library

This page contains no direct external citations. These are the section’s references; they do not independently validate the desk’s proposed scoring or methods.

  1. Together AI — AI Native Cloud →www.together.ai
  2. AI Risk Management Framework — NIST →www.nist.gov
  3. Demystifying evals for AI agents — Anthropic, 9 January 2026 →www.anthropic.com
  4. GroupKFold — scikit-learn documentation →scikit-learn.org
  5. A Field Guide to Rapidly Improving AI Products — Hamel Husain →hamel.dev
  6. Llama 3.3 Community License Agreement — Meta →github.com
  7. Pricing — Together AI, snapshot checked 26 October 2026 →www.together.ai