Comparing Robustness-Oriented Multi-Criteria Decision-Making Methods: A Cross-Domain Benchmark of MEGAN and Recent Aggregation-Based Approaches in Engineering Selection Problems
Abstract
Robustness-oriented multi-criteria decision-making (MCDM) methods aim to produce rankings that remain stable against the choice of normalisation and weighting. Yet, new members of this family are rarely validated independently. This study presents an independent cross-domain benchmark of MEGAN (Multi-criteria Evaluation via Gradual-weighting and Aggregation of Normalised distance matrices) against three established robustness-oriented methods, the Combined Compromise Solution (COCOSO), the Alternative Ranking Order Method Accounting for Two-Step Normalisation (AROMAN), and the Root Assessment Method (RAM), and four conventional reference methods (TOPSIS, VIKOR, WASPAS, EDAS). The benchmark pairs the synthetic renewable energy grid dataset of the original MEGAN study, used for verification, with a real-world flotation machine selection problem. Robustness claims were tested under systematic normalisation and weighting substitution, Monte Carlo weight perturbation, and leave-one-out rank reversal. Agreement proved strongly dataset-dependent: MEGAN aligned with the conventional methods on the flotation problem (Spearman near 0.70) but produced near-opposite rankings on the energy problem (−0.84), and the claimed normalisation and weighting insensitivity was not reproduced, while rank-reversal resistance held fully. A formal analysis shows that the second direction-unification step of MEGAN exactly cancels the normalisation’s direction handling, so every cost criterion is effectively treated as a benefit criterion. Removing this redundant step yields the corrected variant MEGAN-S, which raised the mean rank correlation from −0.29 to +0.60 and from +0.61 to +0.97 on the two problems, while retaining full rank-reversal resistance. Robustness of an MCDM method is thus better understood as a property of the problem structure than a fixed guarantee.