A Machine Learning Approach to Multi-Criteria Decision-Making via Peer-Prediction Trees and Excess Performance Evaluation

Abstract

Multi-criteria decision-making methods conventionally rank alternatives by aggregating normalized criterion values into a composite score, an approach that depends on the normalization scheme, requires externally determined criterion weights, and evaluates performance in absolute terms without accounting for inter-criteria dependencies. This paper proposes machine learning-based Peer-Prediction Trees for multi-criteria decision-making, a novel method that ranks alternatives by their excess performance over data-driven peer expectations. For each criterion, a leave-one-out cross-validated decision tree predicts each alternative’s value from its performance on all other criteria; the residual between observed and predicted values extracts how much the alternative outperforms the empirical trade-off structure constraining its peers. Residuals are standardized, aligned with the preferred criterion direction, and aggregated into an Excess Performance Index using uniqueness-based weights derived endogenously from the cross-validated predictive fit. The method is validated on three real-world datasets: a numeric vehicle selection problem (22 alternatives, five criteria), a fully linguistic financial performance evaluation of Turkish listed firms (32 alternatives, four criteria), and a mixed numeric-linguistic exchange-traded fund selection problem (12 alternatives, nine criteria). The contributions of this work are fourfold: (1) the introduction of peer prediction via leave-one-out cross-validated decision trees as a multi-criteria decision-making ranking paradigm that evaluates excess performance rather than absolute scores; (2) a uniqueness-based criterion weighting scheme embedded into the method, eliminating the need for separate weighting procedures; (3) native handling of numeric, linguistic, and mixed decision matrices within a single unified framework; and (4) empirical validation across three heterogeneous real-world applications demonstrating the method’s versatility and interpretability.