A summarization API is slow under concurrent use.
Clarify performance goals.
Choose two measurements.
Measure p50, p95, p99 latency, timeouts, errors, concurrency, queues, tokens, throughput, and cost together.
Detailed explanation
Long-tail delay is visible.
Bottlenecks can be found.
Quota and contention exist.
Long tails are hidden.
Check official documentation
Compare the following points with the official documentation.
AWS公式AIF-C01 Domain 1.3・3.3のレイテンシ、スループット、同時実行を確認する。
Official references
What to confirm
尾部レイテンシと処理量・費用のトレードオフを測定できる。
Key points
- p95
- Throughput
- Concurrency
Notes
- Environment: AWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認
- Compare the definition with the AIF-C01 objective and the service documentation before applying it.
- Keep the concept separate from a particular answer choice or implementation step.
Foundation review
p95 latency
The response time below which 95 percent of measured requests fall.
Throughput
The amount of work or data completed per unit of time.