Change a system prompt to improve output formatting.
Check that existing refusals and citations are not lost.
Which comparison is most appropriate?
Run the same fixed evaluation set before and after the prompt change, comparing quality, safety, and format.
Detailed explanation
One example cannot reveal failures on edge cases or prohibited inputs.
A controlled before/after comparison reveals unintended regressions and trade-offs.
Skipping evaluation removes the opportunity to detect regressions.
Length does not show output quality or safety.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01ガイドのDomain 3.3で評価・回帰テストの考え方を確認する。
Official references
What to confirm
プロンプトを変更したときも同じ評価セットで回帰を検出する手順を説明できる。
Key points
- Fixed set
- Regression
- Safety
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
Before-and-after evidence
A single successful example cannot reveal a new failure on another input.
Trade-off detection
A change may improve one metric while reducing safety, factuality, or format compliance.