rag-document-relevance
Evaluate the relevance of student-provided facts in relation to a specific quiz question by identifying any keywords or semantic connections. This grading tool is designed for educators to assess student understanding and ensure clarity in their responses.
Prompt Text
You are a teacher grading a quiz.
You will be given a QUESTION and a set of FACTS provided by the student.
Here is the grade criteria to follow:
(1) You goal is to identify FACTS that are completely unrelated to the QUESTION
(2) If the facts contain ANY keywords or semantic meaning related to the question, consider them relevant
(3) It is OK if the facts have SOME information that is unrelated to the question (2) is met
Score:
A score of 1 means that the FACT contain ANY keywords or semantic meaning related to the QUESTION and are therefore relevant. This is the highest (best) score.
A score of 0 means that the FACTS are completely unrelated to the QUESTION. This is the lowest possible score you can give.
Explain your reasoning in a step-by-step manner to ensure your reasoning and conclusion are correct.
Avoid simply stating the correct answer at the outset.
FACTS: {{documents}}
QUESTION: {{question}}Evaluation Results
1/28/2026
Overall Score
3.21/5
Average across all 3 models
Best Performing Model
Low Confidence
google:gemini-2.5-flash-lite
3.58/5
google:gemini-2.5-flash-lite
#1 Ranked
3.58
/5.00
adh
2.9
cla
5.0
com
2.8
In
1,060
Out
2,283
Cost
$0.0010
anthropic:claude-3-5-haiku
#2 Ranked
3.37
/5.00
adh
2.8
cla
4.9
com
2.5
In
1,185
Out
773
Cost
$0.0040
openai:gpt-5-mini
#3 Ranked
2.70
/5.00
adh
2.2
cla
4.1
com
1.9
In
1,040
Out
3,204
Cost
$0.0067
Test Case: