Specialist-model research desk

Expert Q&A

1. “What evidence would make you recommend that we do not fine-tune?”

Frontier Development research deskSection 06 · Page 3 of 17Last updated 26 October 2026

Why it matters: a buyer needs an adviser whose method can produce a no-build result. Ask this before discussing a favourite technique.

A strong answer should cover: baseline optimisation already tried; the remaining failure class; whether it is behavioural, informational or operational; cost of the smallest useful experiment; and a stop rule. Request an anonymised counterexample where simpler work was preferable, with permission to describe it.

Follow-up: “Which observation would reverse your current recommendation?” Bring three sanitised errors and the metric dictionary, not a vague request to make the model better. Error-analysis practice is the grounding reference.

Interview status: Not yet interviewed.

TRACE THE EVIDENCE

Sources cited on this page

  1. Error-analysis practice →hamel.dev