Benchmarking Interactive Multi-Criterion Decision-Making Procedures Following Machine Learning-Based Decision-Maker: Benchmarking Interactive MCDM Procedures

Abstract

Interactive multi-criterion decision-making (iMCDM) procedures rely on a human decision-maker (DM) to iteratively provide preferences, so scalarized subproblems can move toward a preferred Pareto-optimal solution. This human-in-the-loop nature makes systematic benchmarking difficult, as preference information and interaction patterns vary across individuals and problems. To overcome this limitation and to enable computationally-oriented researchers to contribute more profoundly in the MCDM field, we introduce a Machine-based Decision Maker (Machine-DM) that replaces human DMs with pre-trained machine learning models capable of performing the key iMCDM tasks automatically. The Machine-DM predicts objective classifications, such as which objectives should be improved, relaxed, fixed, or allowed to vary and generates corresponding bounding parameters without requiring knowledge of the true target preferred solution. Using this Machine-DM, we develop machine-based versions of four well-known iMCDM procedures: STEM, GUESS, STOM, and NIMBUS. We further propose a set of performance metrics designed to evaluate performance of these iMCDM procedures. Using the Machine-DM concept we also propose a Bench-iMCDM framework for benchmarking iMCDM procedures. Applications to test and engineering problems demonstrate the usefulness of Machine-DM and highlight its potential to serve as a unified framework for comparing a broad class of iMCDM procedures and also to develop new ones.