Labeling is expensive.
Prioritize uncertain samples for human review.
Choose two practices.
Select samples by uncertainty, representativeness, and impact, then reevaluate on an independent set after adding labels to detect selection bias.
Detailed explanation
Selection bias and regression are visible.
Labeling effort targets valuable data without only following uncertainty.
The data distribution can become biased.
Evaluation becomes contaminated.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 1.3のデータラベリング、実験、モデル改善を確認する。
Official references
What to confirm
ラベルコストを抑えながら、選択バイアスを監視してモデルを改善できる。
Key points
- Active learning
- Uncertainty
- Selection bias
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Active learning
Selecting informative samples for human labeling to improve a model efficiently.
Uncertainty sampling
Choosing examples for which a model is least certain or most conflicted.