rag-answer-helpfulness
Evaluate whether RAG answer is helpful to address the question. This is useful because it does not require a ground truth answer.
Prompt Text
You are a teacher grading a quiz.
You will be given a QUESTION and a STUDENT ANSWER.
Here is the grade criteria to follow:
(1) Ensure the STUDENT ANSWER is concise and relevant to the QUESTION
(2) Ensure the STUDENT ANSWER helps to answer the QUESTION
Score:
A score of 1 means that the student's answer meets all of the criteria. This is the highest (best) score.
A score of 0 means that the student's answer does not meet all of the criteria. This is the lowest possible score you can give.
Explain your reasoning in a step-by-step manner to ensure your reasoning and conclusion are correct.
Avoid simply stating the correct answer at the outset.
STUDENT ANSWER: {{student_answer}}
QUESTION: {{question}}Evaluation Results
1/22/2026
Overall Score
3.57/5
Average across all 3 models
Best Performing Model
Low Confidence
google:gemini-1.5-flash
5.00/5
google:gemini-1.5-flash
#1 Ranked
5.00
/5.00
adh
5.0
cla
5.0
com
5.0
anthropic:claude-3-haiku
#2 Ranked
4.43
/5.00
adh
4.5
cla
4.6
com
4.5
GPT-4o Mini
#3 Ranked
1.27
/5.00
adh
1.4
cla
1.5
com
0.9
Test Case:
Tags
langsmith
langchain-ai
StructuredPrompt
QA over documents
English
