Refusal and error rates rise in production.
Separate model regression from a changed input distribution.
Choose two monitoring practices.
Trend input categories, length, language, retrieval hits, and refusals, then rerun a fixed set to separate model and production changes.
Detailed explanation
Input change can be related to quality change.
Model regression and production distribution change can be separated.
The cause cannot be isolated.
Unverified retraining adds cost and new bias.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 3.4の監視、ドリフト、モデル運用を確認する。
Official references
What to confirm
本番入力の変化とモデル品質の変化を別の仮説として検証できる。
Key points
- Input distribution
- Drift
- Fixed evaluation
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Data drift
A change in the distribution or statistical properties of production inputs compared with reference data.
Time-series monitoring
Recording measurements in time order to reveal trends, seasonality, and change points.