tnt-llm-summary-generation
Adapted from: https://arxiv.org/abs/2403.12173
Prompt Text
# Instruction
## Context
- **Goal**: You are tasked with summarizing the input text for the given use case. The summary will represent the input data for clustering in the next step.
- **Data**: Your input data is a conversation history between a User and an AI agent.
# Data
<data>
{content}
</data>
# Questions
## Q1. Summarize the input text in {summary_length} words or less for the use case.
Write the summary between <summary> </summary> tags.
Tips:
- The summary should contain the relevant information for the use case in as much detail as possible.
- Be concise and clear. Do not add phrases like "This is the summary of the data ..." or "Summarized text: ...".
- Similarly, do not reference the user ('the user asked XYZ') unless it's absolutely relevant.
- Within {summary_length} words, include as much relevant information as possible.
- Do not include any line breaks in the summary.
- Provide your answer in **English** only.
## Q2. Explain how you wrote the summary in {explanation_length} words or less.
## Provide your answers between the tags <summary>your answer to Q1</summary>, <explanation>your answer to Q2</explanation>
# Output
Evaluation Results
1/28/2026
Overall Score
2.25/5
Average across all 3 models
Best Performing Model
Low Confidence
anthropic:claude-3-5-haiku
2.33/5
anthropic:claude-3-5-haiku
#1 Ranked
2.33
/5.00
adh
1.6
cla
4.2
com
1.2
In
1,500
Out
522
Cost
$0.0033
google:gemini-2.5-flash-lite
#2 Ranked
2.30
/5.00
adh
1.6
cla
3.9
com
1.4
In
1,415
Out
518
Cost
$0.0003
openai:gpt-5-mini
#3 Ranked
2.12
/5.00
adh
1.6
cla
3.5
com
1.3
In
1,375
Out
3,033
Cost
$0.0064
Test Case:
