A classifier returns confidence 0.8.
Actual accuracy differs by department and input type.
Choose two appropriate evaluations.
Compare predicted and actual accuracy by confidence band and evaluate precision, recall, and business cost at different thresholds.
Detailed explanation
The score can be checked as a probability-like value.
The score can be checked as a probability-like value.
Decision policy impact becomes measurable.
Decision policy impact becomes measurable.
An uncalibrated score is not a probability.
An uncalibrated score is not a probability.
Subgroups and input differences can be hidden.
Subgroups and input differences can be hidden.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01 Domain 3.3の評価指標、信頼度、分類しきい値を確認する。Expected result
内部スコアと校正された確率を区別し、閾値の影響を評価できる。Key points
- Calibration
- Precision
- Recall
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.