The case for adaptation is conditional. Seven primary studies help separate a promising result from a production decision.
Frontier Development · Editorial synthesisPublished 16 October 2026Edition 1.0
Frontier Development interpretation
There is a case for testing. The task decides the case for deployment.
The selected literature demonstrates routes to strong task performance with smaller or adapted models. It also shows why a single favourable result cannot settle the choice. Supervision, model capacity, retained abilities and inference-time computation change what is being compared.
Read these studies as evidence about mechanisms and conditions. The deployment decision still needs a contemporary API comparator, your own protected task evaluation and complete operating costs.
The question this review answers
Which published findings justify an adaptation experiment, and what must that experiment establish?
The question still open
Will a specific candidate outperform the deployable API on your workload today?
Published results · January 2025
The result changes with the task.
The same 7B student exceeds o1-mini on one of these four evaluations. Inspect the full pattern.
Reported evaluation: maximum generation length 32,768 tokens, temperature 0.6 and top-p 0.95. No uncertainty intervals are supplied in this table. Historical author-reported comparison; not a current ranking or a Frontier Development replication.
THE EVIDENCE REGISTER
Read the finding. Inspect its boundary.
Each entry distinguishes the original authors’ finding from our interpretation. The source link takes you to the version used in this review.
Hsieh et al. · Findings of ACL 2023 · pp. 8003–8017
Study setting
T5 students; a 540B PaLM teacher/comparator; e-SNLI, ANLI, CommonsenseQA and SVAMP.
Reported finding
Task-specific training with teacher rationales beat the few-shot PaLM comparator in the reported settings. On ANLI, the 770M T5 student exceeded it using 80% of the labelled training set.
Where it stops
This is a 2023 task-specific comparison. In the unlabelled SVAMP setting, the student needed additional examples to close the gap. It is not a test against today’s APIs.
Our interpretation. Treat task coverage and supervision as experimental variables; do not turn model size into a prediction of quality.
4-bit frozen weights with low-rank adapters; experiments across model scales and instruction datasets.
Reported finding
The authors demonstrated finetuning a 65B model on one 48GB GPU while preserving the full 16-bit finetuning performance of their tested tasks.
Where it stops
A training-memory result does not establish serving cost or quality on an unseen workload. The paper itself cautions against overinterpreting chatbot benchmark scores.
Our interpretation. Use QLoRA as a feasible training candidate, then evaluate the resulting model under the intended production constraints.
Llama-2-7B; code and mathematics; instruction finetuning and continued pretraining.
Reported finding
In the standard low-rank settings studied, LoRA underperformed full finetuning on the target domain while retaining more of the base model’s abilities outside that domain.
Where it stops
This does not directly refute QLoRA: the experiments ask different questions with different tasks and training settings. It does challenge a universal claim of adapter equivalence.
Our interpretation. Compare adaptation strength and retained capabilities; include rank and training method in the experiment rather than assuming they are interchangeable.
Qwen and Llama students distilled from R1; mathematics, science and coding evaluations.
Reported finding
Table 5 reports task-dependent results. R1-Distill-Qwen-7B scores 92.8 on MATH-500 versus 90.0 for o1-mini, but trails it on AIME 2024, GPQA Diamond and LiveCodeBench.
Where it stops
These are the authors’ historical results, not an independent Frontier Development replication. The comparison does not report end-to-end service cost or establish an advantage on customer tasks.
Our interpretation. Preserve the full task profile. A favourable benchmark should not stand in for the acceptance test on the work you actually serve.
Qwen2.5-32B-Instruct, 1,000 curated reasoning examples and inference-time budget forcing.
Reported finding
The paper reports competitive reasoning performance with a small curated training set. Its ablations show that data selection and additional test-time compute influence the result.
Where it stops
The experiment builds on an already pretrained model and teacher-generated reasoning. “1,000 examples” is not the complete training history, and extra inference tokens affect deployment economics.
Our interpretation. Compare quality under a declared latency and token budget. Account for the teacher, data preparation and ongoing inference work.
Preference-trained routers selecting between strong and weak models; public benchmark evaluations.
Reported finding
The study demonstrates improved quality–inference-cost trade-offs through routing, with reported savings above twofold in some settings.
Where it stops
Its token-price objective is not a complete operating-cost model. Routing accuracy, prices and the workload distribution can change; the result does not guarantee production savings.
Our interpretation. Include routing as an alternative to replacing the API entirely. Evaluate the router and exception path as part of the system.
vLLM / PagedAttention serving experiments with controlled model and workload configurations.
Reported finding
The paper reports that KV-cache memory management and scheduling improve serving throughput in its evaluated configurations.
Where it stops
A serving-system throughput comparison is not a present-day API-versus-self-hosting business case. Staffing, review, redundancy and utilisation still require workload-specific inputs.
Our interpretation. Benchmark the serving stack alongside the model. Parameter count alone cannot determine the cost of accepted work.
Different experiments can support different decisions.
These studies do not share a single task, baseline or accounting boundary. Combining their scores into a universal “small versus large” league table would obscure the conditions that produced them.
Training feasibility is one question; adaptation quality is another.
Read the resource-efficiency claim in QLoRA [S2] alongside the target-domain and retention trade-off in LoRA Learns Less [S3]. They motivate a controlled comparison of methods rather than a universal choice of adapter or full finetuning.
The inference budget belongs in the comparison.
s1 [S5] makes test-time computation an explicit variable. A quality comparison is incomplete without the response-time constraint and the work required to produce the answer.
A hybrid may be the relevant alternative.
RouteLLM [S6] motivates testing selective use of the stronger model. The decision need not require replacing every API call.
Serving efficiency and business value use different denominators.
PagedAttention [S7] concerns serving behaviour. Your business comparison must also include accepted completions, review, operating capacity and separate investment.
A proposed evaluation protocol
Make the next test capable of changing the decision.
State a falsifiable claim. Specify the workload, minimum acceptable quality, latency budget and critical failure types.
Pin the alternatives. Record the model versions, prompts, retrieval, decoding settings, adaptation method and a reasonably optimised API comparator.
Protect evaluation material. Separate development and holdout data according to the claim. Include important production segments and document possible contamination.
Measure variability and failures. Record sample sizes and repeated runs where relevant. Justify uncertainty estimates; inspect error severity and grader disagreement.
Price the complete service. Include review, retries, capacity and maintenance. State conditions under which the API or routing option becomes preferable.
Protocol: Frontier Development framework. Reporting principles: NeurIPS checklist. No Frontier Development model experiment is claimed.
Selection. A bounded set of primary studies covering task-specific distillation, low-rank adaptation, inference-time reasoning, routing and serving. Studies were selected for explicit methods and inspectable comparisons. This is not an exhaustive systematic review or a survey of the latest models.
Time boundary. The studies date from 2023–2025. Links and cited passages were checked on 16 October 2026. Where available, URLs pin the exact paper version. A newer model release does not retroactively update these findings.
Accountability. Frontier Development is the publishing desk. Study authors are credited in each source entry. The experiments have not been rerun by Frontier Development; no named external technical reviewer is recorded for this edition.
Version 1.0 · 16 October 2026. Added the evidence register, the Table 5 comparison, applicability limits and a cited brief workflow. Future corrections should identify the claim, source passage and proposed change.