Refusal and error rates rise in production.
Separate model regression from a changed input distribution.
Choose two monitoring practices.
Trend input categories, length, language, retrieval hits, and refusals, then rerun a fixed set to separate model and production changes.
Detailed explanation
Input change can be related to quality change.
Input change can be related to quality change.
Model regression and production distribution change can be separated.
Model regression and production distribution change can be separated.
The cause cannot be isolated.
The cause cannot be isolated.
Unverified retraining adds cost and new bias.
Unverified retraining adds cost and new bias.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01 Domain 3.4の監視、ドリフト、モデル運用を確認する。Expected result
本番入力の変化とモデル品質の変化を別の仮説として検証できる。Key points
- Input distribution
- Drift
- Fixed evaluation
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.