Specialist-model research desk

Direct questions

When is self-hosting cheaper than an API?

Frontier Development research deskSection 01 · Page 7 of 14Last updated 26 October 2026

Only when its whole operating cost divided by accepted output is lower at the workload and quality level you actually need. A low GPU-hour rate is not a cost per task.

Record monthly tasks, input and output lengths, retries, concurrency and demand variation. Price both busy and quiet periods, including capacity needed for failures or peaks. Count review labour, engineering, monitoring, support, storage and data movement once each. Compare recurring economics separately from the one-off migration investment.

Use this definition: recurring cost per accepted task = all monthly operating costs / monthly accepted tasks. If no tasks are accepted, the metric is undefined rather than zero. Compare the same service obligation; a cheap system that abandons difficult cases is not equivalent to a system that resolves them.

An illustrative example, not a forecast: a $40,000 monthly baseline delivering 95,000 accepted tasks costs about $0.421 each. A $25,000 alternative delivering the same number costs about $0.263. A $120,000 initial investment takes eight full operating months to recover if the $15,000 monthly difference persists. Development time, transition costs and price changes can extend that period.

TRACE THE EVIDENCE

Section source library

This page contains no direct external citations. These are the section’s references; they do not independently validate the desk’s proposed scoring or methods.

  1. SFT Trainer — Hugging Face TRL documentation →huggingface.co
  2. Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps — Hugging Face, 3 September 2026 →huggingface.co
  3. A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation — He, Popa and Danilevsky, 19 October 2026 →arxiv.org
  4. Demystifying evals for AI agents — Anthropic, 9 January 2026 →www.anthropic.com
  5. Pricing — Together AI, snapshot checked 26 October 2026 →www.together.ai
  6. Llama 3.3 Community License Agreement — Meta →github.com
  7. A Field Guide to Rapidly Improving AI Products — Hamel Husain →hamel.dev
  8. Guidance on HIPAA & Cloud Computing — HHS Office for Civil Rights →www.hhs.gov
  9. QLoRA: Efficient Finetuning of Quantized LLMs — Dettmers et al., 23 May 2023 →arxiv.org
  10. LoRA Learns Less and Forgets Less — Biderman et al., revision 20 September 2024 →arxiv.org
  11. Extend — Extraction Overview →docs.extend.ai
  12. GroupKFold — scikit-learn documentation →scikit-learn.org
  13. TimeSeriesSplit — scikit-learn documentation →scikit-learn.org
  14. RouteLLM: Learning to Route LLMs with Preference Data — Ong et al., revision 23 February 2025 →arxiv.org
  15. The Open Source AI Definition 1.0 — Open Source Initiative →opensource.org