3.3Standard
Evaluate a new recommendation model.
Compare test accuracy with real-user effects.
Choose two practices.
× Incorrect
Use fixed offline data for candidate comparison and monitor production latency, errors, behavior, complaints, and safety; neither replaces the other.
Detailed explanation
IncorrectChange evaluation data to improve online metrics
The comparison loses credibility.
CorrectCompare accuracy, recall, and safety on fixed offline data
Candidates are compared before full exposure.
CorrectMonitor latency, errors, retention, complaints, and subgroup effects in production
Operational and user impact are measured.
IncorrectHigh offline accuracy eliminates production impact monitoring
Distribution and behavior can differ.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 3.3のモデル評価、運用指標、責任あるAIを確認する。
Official references
What to confirm
開発時指標と本番影響指標を分け、段階展開で確認できる。
Key points
- Offline evaluation
- Production monitoring
- User impact
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Offline evaluation
Comparing model behavior on fixed data and conditions before or without live traffic.
Production metric
A measurement collected during real operation, such as latency, errors, or user impact.
Question IDAWS-AIF-244
EnvironmentAWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
Last technical review2026-08-20
Report an error or rights issue