Hsieh et al. · Findings of ACL 2023 · pp. 8003–8017
Study setting
T5 students; a 540B PaLM teacher/comparator; e-SNLI, ANLI, CommonsenseQA and SVAMP.
Reported finding
Task-specific training with teacher rationales beat the few-shot PaLM comparator in the reported settings. On ANLI, the 770M T5 student exceeded it using 80% of the labelled training set.
Where it stops
This is a 2023 task-specific comparison. In the unlabelled SVAMP setting, the student needed additional examples to close the gap. It is not a test against today’s APIs.
Qwen and Llama students distilled from R1; mathematics, science and coding evaluations.
Reported finding
Table 5 reports task-dependent results. R1-Distill-Qwen-7B scores 92.8 on MATH-500 versus 90.0 for o1-mini, but trails it on AIME 2024, GPQA Diamond and LiveCodeBench.
Where it stops
These are the authors’ historical results, not an independent Frontier Development replication. The comparison does not report end-to-end service cost or establish an advantage on customer tasks.
Llama-2-7B; code and mathematics; instruction finetuning and continued pretraining.
Reported finding
In the standard low-rank settings studied, LoRA underperformed full finetuning on the target domain while retaining more of the base model’s abilities outside that domain.
Where it stops
This does not directly refute QLoRA: the experiments ask different questions with different tasks and training settings. It does challenge a universal claim of adapter equivalence.
Qwen2.5-32B-Instruct, 1,000 curated reasoning examples and inference-time budget forcing.
Reported finding
The paper reports competitive reasoning performance with a small curated training set. Its ablations show that data selection and additional test-time compute influence the result.
Where it stops
The experiment builds on an already pretrained model and teacher-generated reasoning. “1,000 examples” is not the complete training history, and extra inference tokens affect deployment economics.
A current optimised API baseline, a representative protected holdout, subgroup failures, and latency/cost measured under the same acceptance policy.
YOUR ASSESSMENT
Make the reasoning visible to your team.
These are your annotations. The brief preserves opposing evidence and limitations; it does not generate an automatic validation score.
Literature synthesis · source review 16 October 2026. Historical author-reported results, not an independent replication. Downloaded files include the review boundary and all cited sources.