Specialist-model research desk

Comparisons

LoRA/QLoRA versus full fine-tuning: what is the practical trade-off?

Frontier Development research deskSection 03 · Page 3 of 11Last updated 26 October 2026
Dimension Adapter-based approach Full fine-tuning What the buyer should request
Trainable artefact A limited set of added/updated parameters Broad updates to the model Exact saved files and base revision
Resource use Often chosen for lower training memory Can require substantially greater resources Measured peak memory, job cost and failed-run cost
Adaptation capacity Depends on rank, placement, data and task Wider parameter freedom Controlled comparison if quality is limiting
Retention Must still be tested Must still be tested Out-of-task and safety regression set
Portability Adapter must match the base and runtime Checkpoint still has licence/runtime dependencies Import and inference rehearsal elsewhere

QLoRA uses quantisation with low-rank adaptation to reduce training memory. Biderman and colleagues found standard low-rank configurations learned less of certain target domains while forgetting less outside them. These are different studies and questions; one does not invalidate the other. A procurement claim that an adapter is “as good as full fine-tuning” needs task-specific evidence.

Quantisation is also a deployment choice. Retest the actual served precision and runtime rather than assuming the training evaluation transfers unchanged; the PEFT guide explains the relationship between quantisation and adapters but does not certify a buyer’s model.

TRACE THE EVIDENCE

Sources cited on this page

  1. QLoRA →arxiv.org
  2. Biderman and colleagues →arxiv.org
  3. the PEFT guide →huggingface.co