[Paper Review] Interactive Evolutionary Multi-Objective Optimization via Learning-to-Rank

Ke Li, Guiyu Lai+1 more|arXiv (Cornell University)|Apr 6, 2022

Advanced Multi-Objective Optimization Algorithms4 citations

TL;DR

This paper proposes a general interactive evolutionary multi-objective optimization framework that uses learning-to-rank neural networks to model decision maker (DM) preferences from holistic pairwise comparisons. By integrating preference feedback into any existing EMO algorithm via a plug-in module, the method effectively guides the search toward the DM's preferred solutions (SOI), demonstrating robust performance across 48 benchmark problems with up to 10 objectives.

ABSTRACT

In practical multi-criterion decision-making, it is cumbersome if a decision maker (DM) is asked to choose among a set of trade-off alternatives covering the whole Pareto-optimal front. This is a paradox in conventional evolutionary multi-objective optimization (EMO) that always aim to achieve a well balance between convergence and diversity. In essence, the ultimate goal of multi-objective optimization is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory trade-offs among multiple conflicting criteria. Bearing this in mind, this paper develops a framework for designing preference-based EMO algorithms to find SOI in an interactive manner. Its core idea is to involve human in the loop of EMO. After every several iterations, the DM is invited to elicit her feedback with regard to a couple of incumbent candidates. By collecting such information, her preference is progressively learned by a learning-to-rank neural network and then applied to guide the baseline EMO algorithm. Note that this framework is so general that any existing EMO algorithm can be applied in a plug-in manner. Experiments on $48$ benchmark test problems with up to 10 objectives fully demonstrate the effectiveness of our proposed algorithms for finding SOI.

Motivation & Objective

• Address the challenge of identifying decision maker (DM)-preferred solutions (SOI) in high-dimensional multi-objective optimization.
• Overcome the limitations of conventional EMO methods that aim for uniform convergence and diversity across the entire Pareto front.
• Develop a general, algorithm-agnostic framework that integrates human-in-the-loop feedback into any EMO algorithm.
• Reduce cognitive burden on the DM by using indirect preference elicitation (pairwise comparisons) instead of direct preference input.
• Improve search efficiency and accuracy toward SOI by progressively learning and applying DM preferences through machine learning.

Proposed method

• Introduce a three-module framework: consultation (feedback collection), preference elicitation (latent preference modeling), and optimization (guidance via learned preferences).
• Use holistic pairwise comparisons as indirect preference feedback, where the DM selects a preferred solution from two candidates.
• Train a learning-to-rank (LTR) neural network to model the DM’s latent preference from pairwise comparison data.
• Integrate the learned preference model into the environmental selection mechanism of any EMO algorithm (e.g., NSGA-II, MOEA/D, IBEA).
• Apply gradient descent to optimize the LTR model, minimizing ranking loss based on observed pairwise preferences.
• Ensure compatibility with diverse EMO algorithms by translating the learned preference into a tailored selection criterion for each algorithm’s environmental selection process.

Experimental results

Research questions

• RQ1 Can a learning-to-rank model effectively model human preferences from pairwise comparisons in interactive EMO?
• RQ2 How does the proposed framework improve convergence to the DM’s preferred solution (SOI) compared to non-preference-based EMO?
• RQ3 To what extent is the framework robust to noisy or inconsistent preference feedback during interaction?
• RQ4 Can the framework be generalized across different EMO algorithms (Pareto-, indicator-, and decomposition-based) without algorithm-specific design?
• RQ5 How does the level of noise in preference elicitation affect the approximation accuracy of the final solution set?

Key findings

• The proposed I-MOEA/D/LTR, I-NSGA-II/LTR, and I-R2-IBEA/LTR variants consistently converge toward the DM’s preferred solutions across 48 benchmark problems with up to 10 objectives.
• The framework demonstrates robustness to mild preference noise: approximation error remains close to the noiseless 'oracle' case when the noise level (κ) increases.
• I-MOEA/D/LTR is sensitive to high noise levels (low κ), but still maintains acceptable performance, indicating moderate robustness.
• I-NSGA-II/LTR and I-R2-IBEA/LTR show high resilience to noise, suggesting their evolutionary processes are less dependent on precise preference signals.
• The learning-to-rank model effectively captures and applies DM preferences, significantly improving search efficiency toward SOI compared to standard EMO.
• The framework is generalizable and can be seamlessly integrated with diverse EMO algorithms through a plug-in architecture.