Specialist-model research desk

Next steps

Stage 1: can we produce a one-page decision brief?

Frontier Development research deskSection 12 · Page 3 of 12Last updated 26 October 2026

Complete these fields with named owners and dated evidence. An unresolved field is a task, not permission to substitute a supplier’s assumption.

  • □ Task: the input, required output and permitted downstream action.
  • □ Audience: which customers, document families, languages or support intents matter.
  • □ Baseline: current model/service version, prompt, retrieval, parsing and tool permissions.
  • □ Failure: examples grouped by cause, severity and frequency where measured.
  • □ Volume: actual tasks, tokens, concurrency and variation over a representative period.
  • □ Acceptance: critical failures, quality threshold, review policy and latency obligation.
  • □ Economics: recurring cost boundary, proposed investment cap and downside case.
  • □ Ownership: decision owner, data owner, evaluator and eventual service operator.

Use placeholders only inside the working brief when the team must supply a value; do not present those blanks as research findings. If measurement is missing, specify how it will be collected and who will approve it. The brief should fit the decision, with detailed logs attached separately.

TRACE THE EVIDENCE

Section source library

This page contains no direct external citations. These are the section’s references; they do not independently validate the desk’s proposed scoring or methods.

  1. Demystifying evals for AI agents — Anthropic, 9 January 2026 →www.anthropic.com
  2. A Field Guide to Rapidly Improving AI Products — Hamel Husain →hamel.dev
  3. Pricing — Together AI, snapshot checked 26 October 2026 →www.together.ai
  4. GroupKFold — scikit-learn documentation →scikit-learn.org
  5. AI Risk Management Framework — NIST →www.nist.gov
  6. RouteLLM: Learning to Route LLMs with Preference Data — Ong et al., revision 23 February 2025 →arxiv.org
  7. Extend — Extraction Overview →docs.extend.ai
  8. TimeSeriesSplit — scikit-learn documentation →scikit-learn.org
  9. Guidance on HIPAA & Cloud Computing — HHS Office for Civil Rights →www.hhs.gov
  10. The Open Source AI Definition 1.0 — Open Source Initiative →opensource.org
  11. Llama 3.3 Community License Agreement — Meta →github.com