Fraud is only one percent of transactions.
Always predicting normal still gives high accuracy.
Choose two practices.
Use precision, recall, F1, and a confusion matrix, and evaluate thresholds and sampling against the cost of misses and false alarms.
Detailed explanation
Minority-class detection is visible.
Minority-class detection is visible.
Misses and false alarms are balanced.
Misses and false alarms are balanced.
The majority class can hide failures.
The majority class can hide failures.
Important fraud performance disappears.
Important fraud performance disappears.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01 Domain 1.2のクラス不均衡と評価指標を確認する。Expected result
多数クラスに隠れた少数クラスの性能を評価できる。Key points
- Imbalance
- F1
- Confusion matrix
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.