Labeling is expensive.
Prioritize uncertain samples for human review.
Choose two practices.
Select samples by uncertainty, representativeness, and impact, then reevaluate on an independent set after adding labels to detect selection bias.
Detailed explanation
Labeling effort targets valuable data without only following uncertainty.
Labeling effort targets valuable data without only following uncertainty.
Selection bias and regression are visible.
Selection bias and regression are visible.
The data distribution can become biased.
The data distribution can become biased.
Evaluation becomes contaminated.
Evaluation becomes contaminated.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01 Domain 1.3のデータラベリング、実験、モデル改善を確認する。Expected result
ラベルコストを抑えながら、選択バイアスを監視してモデルを改善できる。Key points
- Active learning
- Uncertainty
- Selection bias
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.