Fraud is only one percent of transactions.
Always predicting normal still gives high accuracy.
Choose two practices.
Use precision, recall, F1, and a confusion matrix, and evaluate thresholds and sampling against the cost of misses and false alarms.
Detailed explanation
The majority class can hide failures.
Misses and false alarms are balanced.
Important fraud performance disappears.
Minority-class detection is visible.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 1.2のクラス不均衡と評価指標を確認する。
Official references
What to confirm
多数クラスに隠れた少数クラスの性能を評価できる。
Key points
- Imbalance
- F1
- Confusion matrix
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Class imbalance
A large difference in the number of examples available for different classes.
F1 score
The harmonic mean of precision and recall, balancing the two measures.