Compare two model responses.
Reduce rater bias and record why a response was preferred.
Choose two appropriate practices.
Use the same inputs, anonymize model identity and order, define criteria with examples, and record reasons and conditions.
Detailed explanation
This reduces brand and ordering bias.
This reduces brand and ordering bias.
Raters can apply a more consistent standard.
Raters can apply a more consistent standard.
Extraneous information biases quality judgments.
Extraneous information biases quality judgments.
Results cannot be reproduced or improved.
Results cannot be reproduced or improved.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01ガイドDomain 3.3の人手評価と生成AI品質評価を確認する。Expected result
人手比較評価のバイアス抑制と再現性向上を説明できる。Key points
- Anonymization
- Criteria
- Agreement
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.