[Paper Review] Neural Learning of One-of-Many Solutions for Combinatorial Problems in Structured Output Spaces

Yatin Nandwani, Deepanshu Jindal+2 more|arXiv (Cornell University)|Aug 27, 2020

Image Retrieval and Classification Techniques29 references4 citations

TL;DR

This paper introduces One-of-Many Learning (1oML), a novel framework for training neural networks to solve combinatorial problems in structured output spaces by explicitly modeling solution multiplicity. It proposes a reinforcement learning-based selection module (SelectR) that dynamically chooses the most informative solution from multiple valid outputs during training, significantly improving generalization and achieving up to a 21-point accuracy gain over baselines on Sudoku, N-Queens, and Futoshiki tasks.

ABSTRACT

Recent research has proposed neural architectures for solving combinatorial problems in structured output spaces. In many such problems, there may exist multiple solutions for a given input, e.g. a partially filled Sudoku puzzle may have many completions satisfying all constraints. Further, we are often interested in finding any one of the possible solutions, without any preference between them. Existing approaches completely ignore this solution multiplicity. In this paper, we argue that being oblivious to the presence of multiple solutions can severely hamper their training ability. Our contribution is two fold. First, we formally define the task of learning one-of-many solutions for combinatorial problems in structured output spaces, which is applicable for solving several problems of interest such as N-Queens, and Sudoku. Second, we present a generic learning framework that adapts an existing prediction network for a combinatorial problem to handle solution multiplicity. Our framework uses a selection module, whose goal is to dynamically determine, for every input, the solution that is most effective for training the network parameters in any given learning iteration. We propose an RL based approach to jointly train the selection module with the prediction network. Experiments on three different domains, and using two different prediction networks, demonstrate that our framework significantly improves the accuracy in our setting, obtaining up to 21 pt gain over the baselines.

Motivation & Objective

• To address the critical gap in existing neural models that ignore solution multiplicity in combinatorial problems such as Sudoku and N-Queens.
• To formalize the task of One-of-Many Learning (1oML), where the model must predict any valid solution from a set of possible solutions.
• To improve training efficiency and generalization by dynamically selecting the most informative solution for backpropagation in each training iteration.
• To develop a generic, end-to-end trainable framework compatible with any non-autoregressive prediction network for structured output prediction.
• To demonstrate that solution multiplicity, often treated as noise, can be leveraged as a signal to improve model robustness and accuracy.

Proposed method

• Proposes a novel multiplicity-aware loss (CC-Loss) and identifies its limitations in non-autoregressive models for structured outputs.
• Introduces MinLoss, a greedy baseline that selects the closest ground-truth solution to the current prediction for loss backpropagation.
• Develops I-ExplR, an informed exploration strategy that uses the prediction network to sample diverse solutions beyond the nearest neighbor.
• Proposes SelectR, a reinforcement learning-based selection module that learns a policy to choose solutions that maximize long-term training progress.
• Jointly trains the prediction network and the selection module via RL, enabling exploration of sub-optimal but globally beneficial solutions.
• Employs a policy gradient approach to optimize the selection module, balancing exploration and exploitation in solution space.

Experimental results

Research questions

• RQ1 Can explicit modeling of solution multiplicity improve the generalization and accuracy of neural models for combinatorial problems?
• RQ2 How does solution multiplicity affect the training dynamics and performance of non-autoregressive neural solvers?
• RQ3 Can a learned selection mechanism outperform greedy or random selection of training targets in multi-solution settings?
• RQ4 Does an RL-based selection policy lead to better generalization than heuristic or prediction-driven exploration strategies?
• RQ5 To what extent can a model trained on a subset of all possible solutions still generalize well to inputs with large solution sets?

Key findings

• SelectR achieves up to a 21-point accuracy gain over naive baselines on Sudoku, Futoshiki, and N-Queens, with gains of 13–52 points on multi-solution queries.
• The RL-based SelectR outperforms MinLoss with p-values of 1.00e-16 (N-Queens), 0.03 (Futoshiki), and 1.69e-18 (Sudoku), indicating statistically significant improvement.
• I-ExplR improves over MinLoss on two of three tasks but underperforms SelectR across all domains, highlighting the advantage of RL-based exploration.
• Even when trained on only up to 5 solutions per input (despite larger true solution sets), SelectR achieves significant gains, showing robustness to incomplete solution sets.
• Model performance degrades with increasing solution set size, but SelectR remains the most robust model across all domains.
• The framework is generic and effective across different prediction networks and combinatorial problems, including Sudoku, N-Queens, and Futoshiki.