Specialist-model research desk

Comparisons

Prompting, retrieval, adaptation or routing?

Frontier Development research deskSection 03 · Page 2 of 11Last updated 26 October 2026
Option Best initial hypothesis What changes Buyer still owns Failure or stop signal
Keep current API Current quality and whole-life economics are acceptable Nothing material Monitoring and reassessment A measured constraint emerges
Prompt/schema improvement Instructions or output constraints are weak Request and validation layer Test cases, prompts, retry policy Gains do not survive unfamiliar cases
Retrieval-augmented generation Relevant facts are absent or change frequently Information supplied at inference Corpus permissions, freshness and retrieval tests Correct context is present but behaviour remains wrong
Smaller unadapted model Task is narrower than the model currently used Model and serving configuration Quality gate and fallback Review and failure costs outweigh usage saving
Supervised fine-tuning Stable examples can teach a recurring behaviour Model parameters/adapters Dataset, licences, acceptance and lifecycle Improvement is limited to familiar templates
Distillation Teacher behaviour is valuable and permitted to reuse Student training examples or targets Teacher rights, quality filtering and independence Student copies errors or loses critical behaviours
Preference optimisation/RL A dependable preference or reward signal exists Training objective and feedback loop Reward quality and held-out evaluation Reward increases while real task success deteriorates
Routing/hybrid Traffic contains separable easy and difficult cases Which path handles each request Routing threshold and combined service Misrouting, double-processing or delay negates benefit
Existing specialist API A bounded capability can be bought directly Provider and integration Acceptance, contract and exit design Service cannot handle crucial cases or required controls

TRL’s SFT documentation establishes a concrete implementation route for supervised adaptation. RouteLLM demonstrates one research approach to routing. Neither establishes which row wins for a proprietary workflow.

How should an extraction business use this table?

Begin with the bottleneck: unreadable inputs, missing page structure, semantic field errors, inconsistent formatting or excessive human review. Compare parsing plus a general model, a specialist extraction API and an adapted model using the same complete-document acceptance definition.

Extend’s extraction documentation distinguishes full-document parsing from returning requested fields in a schema. This distinction matters when pricing a service: a parser may still leave extraction, validation and review work to the buyer. A service returning JSON may still need expensive corrections. Count rejected and abstained documents in the original workload denominator.

How should a support platform use it?

Separate intent classification, answer drafting and autonomous resolution into distinct tests. Fresh policy access suggests a retrieval experiment; recurring response-style or tool-selection errors may justify adaptation after the retrieval path is checked. A routing experiment should include unknown intents and deliberate out-of-scope cases.

Test tenant boundaries and permission-sensitive actions. A high average answer score is not an acceptable offset for unauthorised refunds or another customer’s information. The desk recommends treating those as explicit gates rather than hiding them in a weighted average.

EXPLORE THE DECISION

Explore the options

Read the full comparison →
OPTION 01

Keep current API

Best initial hypothesis
Current quality and whole-life economics are acceptable
What changes
Nothing material
Buyer still owns
Monitoring and reassessment
Failure or stop signal
A measured constraint emerges

These are decision hypotheses from the research. The buyer’s own task, rights and acceptance rules determine fit.

TRACE THE EVIDENCE

Sources cited on this page

  1. TRL’s SFT documentation →huggingface.co
  2. RouteLLM →arxiv.org
  3. Extend’s extraction documentation →docs.extend.ai