Human labels are used to train a moderation model.
Labeling instructions changed across teams.
Choose two controls.
Measure agreement and disagreement by subgroup and period, review instructions and sampling, and correct or document systematic label bias.
Detailed explanation
Bias is managed rather than hidden in the model.
Different meanings are incorrectly combined.
Human judgments can be inconsistent or biased.
Systematic labeling differences become visible.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 4.1のラベルバイアス、公平性、ステークホルダーを確認する。
Official references
What to confirm
モデル入力だけでなく、正解ラベルの生成過程も公平性の対象と説明できる。
Key points
- Label bias
- Agreement
- Sampling
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Label bias
Systematic unfairness introduced by the labels, criteria, or process used to define targets.
Participatory review
Including affected people and users in reviewing data, criteria, or system impacts.