Evaluating retrieval relevance

3.3Standard

A retrieval chatbot gives wrong answers.

Investigation shows that unrelated documents are retrieved before generation.

What should be evaluated first?

Hint 1: Focus on the requirement

Separate whether retrieval found useful evidence from whether the generator used that evidence correctly.

Hint 2: Key terms

Review the stages and measures used to evaluate RAG retrieval quality.

Retrieval relevance: The degree to which returned documents address the meaning of a query.
Retrieval recall: The share of useful reference material that the retrieval stage manages to return.
Ranking: Ordering retrieved candidates by estimated usefulness or similarity to a query.
Generation stage: The part of a RAG pipeline that produces an answer from retrieved context.

Question IDAWS-AIF-045

EnvironmentAWS公式AIF-C01試験ガイドとAWS公式ドキュメントの確認

Last technical review2026-08-20

More practice in this area