[Paper Review] Recommending Multiple Criteria Decision Analysis Methods with A New Taxonomy-based Decision Support System

Marco Cinelli, Miłosz Kadziński+3 more|arXiv (Cornell University)|Jun 8, 2021

Multi-Criteria Decision Making4 citations

TL;DR

This paper introduces MCDA-MSS, a novel taxonomy-based decision support system that recommends optimal Multiple Criteria Decision Analysis (MCDA) methods for complex decision-making problems. By evaluating over 200 MCDA methods across problem formulation, preference elicitation, and preference modeling features, the system enhances method selection accuracy, reduces methodological errors, and guides users in refining incomplete problem descriptions, with validation across multiple case studies.

ABSTRACT

We present the Multiple Criteria Decision Analysis Methods Selection Software (MCDA-MSS). This decision support system helps analysts answering a recurring question in decision science: Which is the most suitable Multiple Criteria Decision Analysis method (or a subset of MCDA methods) that should be used for a given Decision-Making Problem (DMP)?. The MCDA-MSS includes guidance to lead decision-making processes and choose among an extensive collection (over 200) of MCDA methods. These are assessed according to an original comprehensive set of problem characteristics. The accounted features concern problem formulation, preference elicitation and types of preference information, desired features of a preference model, and construction of the decision recommendation. The applicability of the MCDA-MSS has been tested on several case studies. The MCDA-MSS includes the capabilities of (i) covering from very simple to very complex DMPs, (ii) offering recommendations for DMPs that do not match any method from the collection, (iii) helping analysts prioritize efforts for reducing gaps in the description of the DMPs, and (iv) unveiling methodological mistakes that occur in the selection of the methods. A community-wide initiative involving experts in MCDA methodology, analysts using these methods, and decision-makers receiving decision recommendations will contribute to expansion of the MCDA-MSS.

Motivation & Objective

• To address the recurring challenge in decision science of selecting the most suitable MCDA method for a given decision-making problem (DMP).
• To develop a comprehensive decision support system capable of recommending from over 200 MCDA methods based on detailed problem characteristics.
• To assist analysts in identifying gaps in DMP descriptions and prioritizing information refinement to improve method suitability.
• To detect and prevent methodological errors in MCDA method selection through systematic evaluation.
• To foster a community-driven expansion of the system by integrating feedback from MCDA experts, practitioners, and decision-makers.

Proposed method

• The system employs a novel taxonomy of MCDA methods based on 15 problem characteristics across four dimensions: problem formulation, preference elicitation, preference model features, and recommendation construction.
• It uses a structured decision model that maps DMP attributes to method capabilities, enabling automated method recommendations.
• The system evaluates method applicability by comparing DMP specifications against predefined criteria, including types of input data, preference intensity, and required output formats.
• It includes a gap-analysis module that identifies missing or ambiguous DMP attributes and suggests priority improvements.
• The recommendation engine integrates a multi-criteria filtering process that ranks methods by compatibility and robustness.
• The system is implemented as a web-based tool with extensible architecture to support community contributions and method updates.

Experimental results

Research questions

• RQ1 Which MCDA method is most appropriate for a given decision-making problem based on its structural and preference-related characteristics?
• RQ2 How can a decision support system effectively guide users in refining incomplete or ambiguous problem descriptions to improve method selection?
• RQ3 What are the key features of a taxonomy that enable accurate and scalable recommendation of MCDA methods?
• RQ4 How can methodological errors in MCDA selection be systematically detected and prevented?
• RQ5 What role can community feedback play in enhancing the coverage and reliability of a method recommendation system?

Key findings

• The MCDA-MSS successfully recommends appropriate MCDA methods for a diverse set of case studies, including both simple and highly complex decision problems.
• The system identifies and highlights missing or inconsistent information in DMP descriptions, enabling users to prioritize data collection and improve method suitability.
• The system detects methodological inconsistencies in MCDA method selection, such as mismatched preference modeling requirements, reducing the risk of flawed recommendations.
• The taxonomy-based approach enables systematic coverage of over 200 MCDA methods, ensuring broad methodological reach.
• The system demonstrates robustness in recommending methods even when no exact match exists in the method collection, by suggesting alternative or hybrid approaches.
• Community feedback mechanisms are shown to be effective in expanding method coverage and improving system accuracy over time.