A churn model uses a field recorded only after a customer has churned.
The offline score is high, but the field is unavailable at prediction time.
Which mitigation is appropriate?
Use only data available at prediction time and design time-aware splits so future information cannot enter training.
Detailed explanation
This prevents future information from making the offline score unrealistically high.
This prevents future information from making the offline score unrealistically high.
Creating unavailable information is not a valid production design.
Creating unavailable information is not a valid production design.
This removes evaluation independence and worsens leakage.
This removes evaluation independence and worsens leakage.
The high score may be caused by leakage.
The high score may be caused by leakage.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01ガイドDomain 1.3のデータ品質と評価設計を確認する。Expected result
予測時点、将来情報、時間順分割の関係を説明できる。Key points
- Future information
- Prediction time
- Leakage
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.