Preventing evaluation leakage

Review test isolation and data leakage.

Data leakage

Information unavailable at prediction time that accidentally enters training or evaluation.

Holdout test

Data kept separate from model selection for a final performance estimate.

Entity split

Separating records by customer, device, or other entity to prevent cross-entity leakage.

Temporal split

Using earlier periods for development and later periods for evaluation.