CFX-Bench: A Benchmark for Counterfactual Explanations

Abstract

Counterfactual explanations support model interpretability by identifying actionable recourse strategies, yet their evaluation remains fragmented across metrics, baselines, and experimental protocols. This paper introduces CFX-Bench, a unified benchmarking suite for comparing counterfactual explanations for tabular classification across datasets, models, and explainers. The framework integrates quantitative and qualitative metrics, local and global explanation methods, and LLM-based components for verbalization and human-centered assessment. By combining standardized benchmarking with semantic evaluation, CFX-Bench provides a reproducible environment for analyzing the quality, reliability, and coherence of counterfactual explanations, while supporting the development of new explanation methods.