AWS Certified AI Practitioner (AIF-C01) · Objective 3.3
Evaluating harmful output
3.3Standard
Test a public chatbot before release.
Include normal, adversarial, and boundary prompts.
Choose two appropriate tests.
Hint 1: Focus on the requirement
Test safety controls with both ordinary harmful requests and attempts to evade them, and assess refusal quality rather than refusal rate alone.
Hint 2: Key terms
Review safety evaluation and adversarial inputs.
Safety evaluation
Measuring whether a system handles harmful, dangerous, or prohibited requests according to policy.
Adversarial input
An input intentionally crafted to expose weaknesses or bypass a model's controls.
Jailbreak
A prompt pattern intended to make a model ignore its rules or safety instructions.
Refusal quality
Whether a refusal is appropriate, clear, safe, and accompanied by a useful benign alternative when possible.
× Incorrect
Use representative harmful categories and evasive variants, then check correct refusal, safe alternatives, language variation, and severe individual failures.
Detailed explanation
A
CorrectCheck refusal correctness, safe alternatives, language variants, and retries
Useful safety is measured beyond a refusal count.
B
IncorrectIgnore severe cases because the category average is good
A few failures can still cause serious harm.
C
IncorrectNever try a transformed version after one refusal
Evasion inputs may be missed.
D
CorrectPrepare harmful, discriminatory, self-harm, and personal-data requests
Safety-policy boundaries can be tested.
Check official documentation
Compare the following points with the official documentation.
Path & Pass is an independent learning site and is not affiliated with any certification body. Questions are original practice content based on published objectives.
