Change a system prompt to improve output formatting.
Check that existing refusals and citations are not lost.
Which comparison is most appropriate?
Run the same fixed evaluation set before and after the prompt change, comparing quality, safety, and format.
Detailed explanation
A controlled before/after comparison reveals unintended regressions and trade-offs.
A controlled before/after comparison reveals unintended regressions and trade-offs.
One example cannot reveal failures on edge cases or prohibited inputs.
One example cannot reveal failures on edge cases or prohibited inputs.
Length does not show output quality or safety.
Length does not show output quality or safety.
Skipping evaluation removes the opportunity to detect regressions.
Skipping evaluation removes the opportunity to detect regressions.
Try it yourself
An example you can run in a temporary verification environment.
AWS公式AIF-C01ガイドのDomain 3.3で評価・回帰テストの考え方を確認する。Expected result
プロンプトを変更したときも同じ評価セットで回帰を検出する手順を説明できる。Key points
- Fixed set
- Regression
- Safety
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.