AWS Certified AI Practitioner (AIF-C01) · Objective 3.4
Building a model benchmark
3.4Standard
Several model candidates must be compared.
Avoid one score.
Choose two designs.
Hint 1: Focus on the requirement
Compare models on the same representative, boundary, safety, multilingual, and long-context cases with matched quality and cost measures.
Hint 2: Key terms
Review model benchmarks.
Benchmark
A repeatable evaluation used to compare models under defined conditions.
Representative case
An example reflecting expected users, inputs, and operating conditions.
Safety case
A test designed to measure policy compliance, harmful behavior, or refusal quality.
Matched evaluation
A comparison using the same data, criteria, and environment for each candidate.
× Incorrect
Use a fixed set of representative, boundary, safety, multilingual, long-context, and refusal cases and compare quality, grounding, safety, latency, cost, and data terms.
Detailed explanation
A
IncorrectChoose the production model from the highest public score
Tasks and conditions differ.
B
CorrectInclude representative, boundary, safety, multilingual, long-context, and refusal cases in a fixed set
Limits and needs are tested.
C
CorrectCompare quality, evidence, safety, latency, cost, and data-use terms under equal conditions
Selection is practical.
D
IncorrectUse a different evaluation set for each model
Comparison is unfair.
Check official documentation
Compare the following points with the official documentation.
Path & Pass is an independent learning site and is not affiliated with any certification body. Questions are original practice content based on published objectives.