Specialist-model research desk

Expert Q&A

3. “How should we balance learning the target task against forgetting other behaviour?”

Frontier Development research deskSection 06 · Page 5 of 17Last updated 26 October 2026

Why it matters: a specialist that improves the main metric may become less reliable on exceptions, refusals or instructions outside the training distribution.

A strong answer should cover: adaptation capacity, rank choices, data mixture, base-model behaviour and retention tests. Ask the expert to separate demonstrated findings from plausible mechanisms. The learning/forgetting study is a precise starting point, not a universal forecast.

Follow-up: “Which retained capabilities are mandatory for this product, and how would you test them without making the evaluation unmanageably broad?” Require a shortlist of material checks, not an indiscriminate benchmark bundle.

Interview status: Not yet interviewed.

TRACE THE EVIDENCE

Sources cited on this page

  1. The learning/forgetting study →arxiv.org