Should we contact suppliers before we have a dataset?
You can ask about scope and fit using non-sensitive metadata, but do not mistake that conversation for feasibility evidence. Build a protected evaluation before an award based on performance; evaluation design guidance is a useful starting point.
When should we stop a pilot?
Stop when a critical gate fails, permission remains unresolved or the agreed experiment cannot establish a worthwhile result. An error-led approach can help decide whether the next intervention should address data, prompts or workflow instead.
Is self-hosting a required next step?
No: compare managed and self-operated routes against the actual service, rights and cost constraints. Infrastructure list prices alone cannot establish which route has lower cost per accepted task.
Can the supplier control the final test?
It can help design the rubric, but preserve buyer-side acceptance and protect final answers from development. Group-aware splitting is one available mechanism within a broader leakage-control policy.
What should we reassess after launch?
Reassess critical quality, review burden, reliability, accepted-task cost and the ability to exit. NIST’s lifecycle framework supports continuing risk management, while the exact triggers should reflect the buyer’s system.