Specialist-model research desk

Saved workspace
Readiness quiz

1. Can we define one business task and its failure consequences?

Frontier Development research deskSection 08 · Page 3 of 20Last updated 26 October 2026

A — 0: We want “better AI” or lower tokens, without an accepted-output definition. B — 1: The workflow is chosen, but acceptance and important errors are not agreed. C — 2: Product and operations have signed a metric dictionary defining accepted, rejected, escalated and critical-failure outcomes.

Evidence to attach: task contract and three sanitised examples with expected outcomes. If weak: narrow to one workflow before testing methods. An invoice field and a fully accepted invoice are different units.

Question 1 of 12

Can we define one business task and its failure consequences?

Evidence to show: task contract and three sanitised examples with expected outcomes.

Read the full question →

A desk-designed readiness rubric, not a validated prediction of success. Progress stays in this browser; no answers are sent to the desk.

TRACE THE EVIDENCE

Section source library

This page contains no direct external citations. These are the section’s references; they do not independently validate the desk’s proposed scoring or methods.

  1. Demystifying evals for AI agents — Anthropic, 9 January 2026 →www.anthropic.com
  2. Llama 3.3 Community License Agreement — Meta →github.com
  3. AI Risk Management Framework — NIST →www.nist.gov
  4. RouteLLM: Learning to Route LLMs with Preference Data — Ong et al., revision 23 February 2025 →arxiv.org
  5. A Field Guide to Rapidly Improving AI Products — Hamel Husain →hamel.dev
  6. GroupKFold — scikit-learn documentation →scikit-learn.org
  7. Pricing — Together AI, snapshot checked 26 October 2026 →www.together.ai
  8. The Inference Engineering Masterclass — Philip Kiely and Ali Taha, Latent Space, 3 August 2026 →latent.space
  9. The Open Source AI Definition 1.0 — Open Source Initiative →opensource.org
  10. SB26-189 Automated Decision-Making Technology — Colorado General Assembly →leg.colorado.gov
  11. Guidance on HIPAA & Cloud Computing — HHS Office for Civil Rights →www.hhs.gov