A voice assistant must answer within 200 ms.
Maintain quality while meeting the target.
Choose two comparisons.
Measure p50 and p95 latency, output length, quality, and safety under the same inputs, then evaluate smaller models, streaming, and caching.
Detailed explanation
Slow tail requests are visible.
Slow tail requests are visible.
Several configurations can meet the requirement.
Several configurations can meet the requirement.
Tail latency can be higher.
Tail latency can be higher.
User impact and risk increase.
User impact and risk increase.
Try it yourself
An example you can run in a temporary verification environment.
Amazon Bedrock公式モデル選択、推論性能、AIF-C01 Domain 3.1を確認する。Expected result
平均値だけでなくテール遅延と品質・安全性を含めてモデルを選べる。Key points
- p95
- Streaming
- Quality
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Command output formatting can vary slightly by distribution or tool version.
- Run the example in a temporary directory or process when possible.
Foundation review
Read the scope first
Check whether the command acts on the current shell, a new process, an existing process, or a file.
Verify the observable result
Use the supplied command and compare the output with the expected result.