Compare a new model with the production model.
Keep user responses on the current model.
Choose two practices.
Send a copy of production inputs to the new model without serving its response, while controlling privacy, cost, latency, and stop conditions.
Detailed explanation
Production-like data can be evaluated safely.
Production-like data can be evaluated safely.
The comparison is operationally controlled.
The comparison is operationally controlled.
That is not a shadow deployment.
That is not a shadow deployment.
Comparison and accountability are lost.
Comparison and accountability are lost.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01 Domain 1.3のデプロイ方式とモデル評価を確認する。Expected result
本番相当の入力で新モデルを安全に比較できる。Key points
- Shadow
- Comparison
- Stop condition
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.