Split by the unit that could leak information, not automatically by row. Depending on the product, that unit may be a customer, template family, conversation, repository or time period.
Keep related documents and derived examples together. Record hashes, source identifiers and synthetic-data lineage; compare exact duplicates and possible semantic near-duplicates across splits. Protect holdout answers from supplier training, prompt optimisation and teacher-model generation. A clean identifier scan alone is insufficient.
Use a chronological test for future workload performance where appropriate, and explain that a new-customer test answers a different question from a future-month test. scikit-learn’s TimeSeriesSplit documentation explains why ordinary splitting can train on future observations. Report the actual split policy, not just “80/20”.