Specialist-model research desk

Scorecard

How should the buyer use it from shortlist to renewal?

Frontier Development research deskSection 09 · Page 9 of 13Last updated 26 October 2026

At shortlist, use scope and evidence availability to decide who receives the common brief. At pilot award, score the plan and credible delivery evidence, explicitly marking future performance as unproven. At production acceptance, replace plan-based ratings with measured results. At renewal, reassess costs, reliability and exit feasibility against an updated baseline.

Keep an evidence register with criterion, claim, source passage/file, inspection date, rating, coverage flag, reviewer and next action. Record conflicts and supplier payments separately from scoring. A non-participating provider remains “not assessed”, not “bad”.

Close the process with a signed decision memo explaining the selected option, rejected alternatives, conditions and reassessment trigger. The useful output is an accountable decision, not an impressive-looking number.

TRACE THE EVIDENCE

Section source library

This page contains no direct external citations. These are the section’s references; they do not independently validate the desk’s proposed scoring or methods.

  1. Together AI — AI Native Cloud →www.together.ai
  2. AI Risk Management Framework — NIST →www.nist.gov
  3. Demystifying evals for AI agents — Anthropic, 9 January 2026 →www.anthropic.com
  4. GroupKFold — scikit-learn documentation →scikit-learn.org
  5. A Field Guide to Rapidly Improving AI Products — Hamel Husain →hamel.dev
  6. Llama 3.3 Community License Agreement — Meta →github.com
  7. Pricing — Together AI, snapshot checked 26 October 2026 →www.together.ai