Classify the failure first
Collect a small, sanitised set of failed tasks and inspect why each failed. Missing or stale information points to a different intervention from a consistent formatting error. Record the evidence that led to the classification. Do not infer a training requirement merely because the data is proprietary.
Test access to the right information
For information failures, inspect source coverage, document freshness, retrieval relevance and the instructions that connect retrieved material to the response. A retrieval experiment should test whether the needed evidence is available and whether the model uses it correctly. Judge the final task, including unsupported statements and unresolved questions.
Test stable behaviour separately
For repeated schema or style failures, first strengthen the prompt and examples. If those changes plateau, a bounded adaptation experiment may be useful when rights, labels and independent evaluation are ready. Fine-tuning is not a substitute for a reliable source of rapidly changing facts. Hybrid configurations can be appropriate, but each added component needs an owner and evidence of value.
Keep the experiment narrow
Change one consequential part of the workflow at a time and preserve the baseline configuration. Compare task-level acceptance, failure slices, latency and operating effort. Stop adaptation if a simpler retrieval or prompt change resolves the constraint. The decision map records the proposed intervention, its owner, the cheapest test and the evidence that would reverse it.
Bring to the decision review
Bring three sanitised failures, the source material available at the time and the correct accepted outcome. Ask the team to explain whether the missing ingredient was knowledge, instruction following, reasoning or an uneconomic exception path. Where several causes coexist, test them separately. A purchasing decision should not depend on a broad label such as “needs a custom model” when the actual failure can be investigated more narrowly.
Sources & editorial record
This is Frontier Development’s decision framework. The references below support the general technical concepts; the worked scenarios and procurement questions are our analysis.
References accessed 16 October 2026. No current price, product availability or legal clearance is asserted. Published study findings are attributed above; this guide reports no Frontier Development experiment. Read the comparison method.
Review history and research boundary
Edition 1.1 · 16 October 2026: added claim-level attribution, explicit evidence boundaries and links to the flagship synthesis. Published by Frontier Development. No named external technical reviewer is recorded. This guide provides a decision framework; it is not a Frontier Development experimental result.
Read the primary-study synthesis →Open the architecture tool
Explore a complete synthetic example before using your own inputs.