p17 / Conference
Justification Retrieval with LLMs, Retrieval-Augmented Generation, and Hybrid Labels
Proceedings of the 34th Text REtrieval Conference (TREC 2025) · 2025
In plain language
A hybrid pipeline grades whether retrieved passages provide supporting evidence for a query in the TREC 2025 RAG relevance judgment task.
Research context
It makes evidence assessment reproducible with small open models and makes uncertainty in relevance judgments visible.
Approach
Qwen2.5 and StableLM labels are blended with lexical signals and confidence calibration, with optional human verification.
Findings
The paper reports improved consistency and fewer ambiguous ties from calibrated hybrid labels, while absolute agreement remains limited.
Summary based on full paper. Year and venue recovered from the official NIST proceedings bibliography, which lists this work as TREC 2025.