Back to prompts

self-rag-retrieval-grader

extraction0 savesSource

Assess the relevance of a retrieved document to a user question by providing a binary score of 'yes' or 'no', based on the presence of related keywords or semantic meaning. This tool is useful for filtering out irrelevant documents in information retrieval systems.

Prompt Text

You are a grader assessing relevance of a retrieved document to a user question. It does not need to be a stringent test. The goal is to filter out erroneous retrievals. If the document contains keyword(s) or semantic meaning related to the user question, grade it as relevant.

Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question.

Retrieved document: {document}

User question: {question}

Evaluation Results

1/28/2026
Overall Score
2.91/5

Average across all 3 models

Best Performing Model
Low Confidence
google:gemini-2.5-flash-lite
4.51/5
google:gemini-2.5-flash-lite
#1 Ranked
4.51
/5.00
adh
4.3
cla
5.0
com
4.2
In
485
Out
5
Cost
$0.0001
openai:gpt-5-mini
#2 Ranked
2.47
/5.00
adh
1.7
cla
4.2
com
1.5
In
485
Out
1,517
Cost
$0.0032
anthropic:claude-3-5-haiku
#3 Ranked
1.77
/5.00
adh
0.8
cla
4.0
com
0.5
In
550
Out
497
Cost
$0.0024
Test Case:

Tags