International Workshop on Beyond Relevance-based EValuation of RAG Systems (BREV-RAG) 2025 (BREV-RAG)

BREV-RAG 2025
BREV-RAG 2025: Beyond Relevance-based EValuation of RAG Systems

Proceedings of the International Workshop on Beyond Relevance-based EValuation of RAG Systems (BREV-RAG) 2025
co-located with 3rd International ACM SIGIR Conference on Information Retrieval in the Asia Pacific (SIGIR-AP 2025)

Xi'an, China, December 10th, 2025.

Edited by

Tetsuya Sakai, Waseda University, Japan
Sijie Tao, Waseda University, Japan
Zhicheng Dou, Renmin University of China, China
Junjie Wang, Tsinghua University, China
Haoxiang Shi, Inner Mongolia University of Technology, China
Nuo Chen, The Hong Kong Polytechnic University, HK, China
Atsushi Keyaki, Hitotsubashi University , Japan

Overview of Papers Contributed to BREV-RAG@SIGIR-AP 20251-4
Tetsuya Sakai, Sijie Tao, Zhicheng Dou, Junjie Wang, Haoxiang Shi, Nuo Chen, Atsuhi Keyaki
Evaluating the Modesty of RAG Systems at the NTCIR-19 R2C2 Task: Potential Challenges5-21
Tetsuya Sakai, Sijie Tao, Atsuya Ishikawa, Hanpei Fang, Ziliang Zhao, Yujia Zhou, Nuo Chen, Young-In Song
Aspect-based Evaluation of Personalization and Diversification in Conversational Search Systems22-33
Risa Hiramatsu, Rikiya Takehi, Tetsuya Sakai
Retrieval-Augmented Relevance Judgment for Specialized Domains34-46
Taichi Motegi, Makoto P. Kato, Kazuhisa Hatakeyama, Yosuke Yurikusa
Measuring Group Fairness Differences in RAG Pipelines: A Pilot Study47-58
Sijie Tao, Tetsuya Sakai
RAG System for Supporting Japanese Litigation Procedures: Faithful Response Generation Complying with Legal Norms59-65
Yuya Ishihara, Atsushi Keyaki, Hiroaki Yamada, Ryutaro Ohara, Mihoko Sumida

We offer a BibTeX file for citing papers of this workshop from LaTeX.

2025-11-12: submitted by Nuo Chen, metadata incl. bibliographic data published under Creative Commons CC0
yyyy-mm-dd: published on CEUR-WS.org |valid HTML5|

BREV-RAG 2025BREV-RAG 2025: Beyond Relevance-based EValuation of RAG Systems

Proceedings of the International Workshop on Beyond Relevance-based EValuation of RAG Systems (BREV-RAG) 2025 co-located with 3rd International ACM SIGIR Conference on Information Retrieval in the Asia Pacific (SIGIR-AP 2025)

Xi'an, China, December 10th, 2025.

Table of Contents

BREV-RAG 2025
BREV-RAG 2025: Beyond Relevance-based EValuation of RAG Systems

Proceedings of the International Workshop on Beyond Relevance-based EValuation of RAG Systems (BREV-RAG) 2025
co-located with 3rd International ACM SIGIR Conference on Information Retrieval in the Asia Pacific (SIGIR-AP 2025)