3.4Standard
Humans grade generated summaries.
Reduce evaluator variation.
Choose two practices.
× Incorrect
Use a rubric for accuracy, completeness, evidence, readability, and safety, with training, examples, blind review, multiple graders, and adjudication.
Detailed explanation
CorrectUse a rubric defining accuracy, completeness, evidence, readability, and safety
Evaluation criteria are aligned.
IncorrectLet every evaluator use different criteria
Results cannot be compared.
IncorrectSave only the mean and not disagreements
Improvement evidence is lost.
CorrectProvide training, examples, blind review, multiple graders, and adjudication
Subjective variation can be measured and reduced.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 3.4の人手評価、品質基準、生成AI評価を確認する。
Official references
What to confirm
評価者間で一貫した品質評価を行い、不一致も改善へ利用できる。
Key points
- Rubric
- Blind review
- Agreement
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Rubric
A table of evaluation criteria, levels, and examples used for consistent judgments.
Blinded evaluation
Reviewing outputs without seeing model or condition labels that could bias judgment.
Question IDAWS-AIF-358
EnvironmentAWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
Last technical review2026-08-20
Report an error or rights issue