Use sanitised examples and a time-bounded exercise; do not solicit unpaid production work. The following prompts are proposed assessment tools.
- Give a dataset with repeated customer templates. Ask the candidate to explain the split, likely leakage and what the result would generalise to.
- Give a model with better field accuracy but worse document acceptance. Ask which decision metric matters and what investigation follows.
- Give two price sheets and a changing traffic pattern. Ask for the missing inputs, not an invented ROI figure.
- Give an adapter and an incomplete model card. Ask what else is needed for a reproducible handover.
- Give conflicting human and LLM-judge labels. Ask for a calibration and adjudication process.
- Give a failed deployment. Ask how they would roll back without losing auditability.
Strong answers state assumptions and uncertainty. A candidate who recommends fixing the baseline or stopping an adaptation project may show better judgement than one who always proposes a more complex training method.