[Paper Review] Phase transition in the knapsack problem

Nitin Yadav, Carsten Murawski+2 more|arXiv (Cornell University)|Jun 26, 2018

Optimization and Search Problems17 references3 citations

TL;DR

This paper investigates the phase transition in the 0-1 Knapsack problem, identifying normalized capacity and profit as key parameters that determine solvability and computational hardness. It demonstrates that both computational difficulty and human decision-making time peak near the phase transition boundary (c/p ≈ 1), providing empirical and theoretical evidence that this region represents the most challenging instances for both machines and humans.

ABSTRACT

We examine the phase transition phenomenon for the Knapsack problem from both a computational and a human perspective. We first provide, via an empirical and a theoretical analysis, a characterization of the phenomenon in terms of two instance properties; normalised capacity and normalised profit. Then, we show evidence that average time spent by human decision makers in solving an instance peaks near the phase transition. Given the ubiquity of the Knapsack problem in every-day life, a better understanding of its structure can improve our understanding not only of computational techniques but also of human behavior, including the use and development of heuristics and occurrence of biases.

Motivation & Objective

• To characterize the phase transition in the Knapsack problem using normalized capacity and profit as order parameters.
• To determine whether computational hardness and human cognitive difficulty coincide near the phase transition boundary.
• To provide theoretical and empirical evidence linking instance complexity to the solvability phase transition in both algorithmic and human decision-making contexts.
• To explore the implications of phase transition for understanding heuristics, biases, and cognitive overload in human problem-solving.

Proposed method

• Empirically analyze random 0-1 Knapsack instances using normalized capacity (c) and normalized profit (p) as key instance properties.
• Theoretically derive a lower bound on solution probability using a fast greedy-type estimation rule to model solvability.
• Conduct human experiments where participants solve Knapsack instances via a computer interface, measuring time spent per search state.
• Use log(c/p) as a proxy for proximity to the phase transition, and correlate it with average time spent per state by human participants.
• Compare human response times with Minizinc solver performance on identical instances to validate the phase transition effect.
• Apply the constrainedness measure κ = 1 − log₂(E[Sol])/N to estimate phase transition region, showing κ ≈ 1 when c/p ≈ 1.

Experimental results

Research questions

• RQ1 Does the 0-1 Knapsack problem exhibit a phase transition in solvability, and if so, what are the key instance parameters that define it?
• RQ2 Are computationally hard instances of the Knapsack problem located near the phase transition boundary, as observed in other NP-complete problems?
• RQ3 Does human decision-making time peak near the phase transition, indicating increased cognitive difficulty?
• RQ4 How does the constrainedness measure κ relate to the phase transition in the Knapsack problem?
• RQ5 To what extent do human reasoning patterns and computational hardness align near the phase transition in the Knapsack problem?

Key findings

• The phase transition in the Knapsack problem occurs near log(c/p) ≈ 0, corresponding to c/p ≈ 1, where normalized capacity and profit are balanced.
• Computational hardness peaks at the phase transition boundary, consistent with findings in other NP-complete problems such as SAT and Hamiltonian Circuit.
• Human participants spend the longest average time per search node in the vicinity of the phase transition, indicating increased cognitive effort.
• The average time spent by humans on nodes near the phase transition (log(c/p) ≈ 0) exceeds that at both lower and higher values, confirming a peak in human difficulty.
• The constrainedness measure κ ≈ 1 when c/p ≈ 1, confirming that this region corresponds to the critical point of the phase transition.
• The study provides empirical and theoretical evidence that the phase transition in solvability is a unifying factor for both algorithmic and human problem-solving difficulty in the Knapsack problem.