TE4 RS: Toward End-to-End Explanation Evaluation for Recommender Systems
Abstract
Explanations in Recommender Systems (RS) play a crucial role in enhancing transparency, trust, and user satisfaction. Benchmarking such explanations, however, is not trivial, as existing evaluation frameworks typically do not assess all explanation stages (i.e., generation, abstraction, formatting, and communication) or evaluate them in isolation. In this paper, we present TE4 RS, an end-to-end explanation evaluation framework for RS that assesses explanation methods across all four stages: generation, abstraction, format, and communication. It contains 20 explanation methods covering seven categories and 31 evaluation metrics. We also integrate large language models to format the explainers' abstract outputs and to serve as proxy evaluators for human evaluation. TE4 RS allows to set up an evaluation pipeline that mimics a specific application scenario and compares multiple explanation methods, thereby fostering a holistic, structured evaluation of RS explanations. The code can be found at: https://gitlab.ifi.uzh.ch/DDIS-Public/te4-rs.