[Paper Review] Quantum POMDPs

Jennifer Barry, Daniel T. Barry+1 more|arXiv (Cornell University)|Jun 11, 2014

Quantum Computing Algorithms and Architecture13 references70 citations

TL;DR

This paper introduces Quantum Observable Markov Decision Processes (QOMDPs), the quantum analog of classical POMDPs, where belief states are quantum states evolved via superoperators. The key contribution is proving that while goal-state reachability is decidable in POMDPs, it becomes undecidable in QOMDPs due to quantum state superposition and entanglement, highlighting a fundamental computability gap between classical and quantum partially observable decision processes.

ABSTRACT

We present quantum observable Markov decision processes (QOMDPs), the quantum analogues of partially observable Markov decision processes (POMDPs). In a QOMDP, an agent's state is represented as a quantum state and the agent can choose a superoperator to apply. This is similar to the POMDP belief state, which is a probability distribution over world states and evolves via a stochastic matrix. We show that the existence of a policy of at least a certain value has the same complexity for QOMDPs and POMDPs in the polynomial and infinite horizon cases. However, we also prove that the existence of a policy that can reach a goal state is decidable for goal POMDPs and undecidable for goal QOMDPs.

Motivation & Objective

• . The paper aims to formalize a quantum generalization of POMDPs, where belief states are quantum states rather than classical probability distributions.
• It investigates the computational complexity and decidability of decision problems in the quantum setting, particularly comparing them to classical POMDPs.
• The research seeks to understand whether quantum control and reasoning in uncertain environments differ fundamentally in computability from classical counterparts.
• It aims to establish foundational results for quantum decision-making under partial observability, relevant to quantum control and fault-tolerance.

Proposed method

• . QOMDPs are defined as a tuple ⟨S, A, T, O, R, γ⟩ where states are quantum states, actions are superoperators, and observations are positive operator-valued measures (POVMs).
• The belief state evolves via quantum operations (superoperators), generalizing the Bayesian update in classical POMDPs.
• The paper reduces goal-state reachability in POMDPs to a finite-state MDP problem using tree-based policy representations and graph reachability analysis.
• It proves undecidability in QOMDPs by reduction from the matrix mortality problem, showing that quantum evolution can simulate undecidable computational processes.
• The analysis leverages quantum state superposition and entanglement to construct systems where reachability cannot be algorithmically determined.
• A key technical tool is the use of quantum state trees and the mapping of classical policy trees to quantum belief states via measurement and superoperator evolution.

Experimental results

Research questions

• RQ1 . Is goal-state reachability decidable in QOMDPs, given that it is decidable in classical POMDPs?
• RQ2 . What is the computational complexity of policy existence in QOMDPs compared to POMDPs in both finite and infinite horizon cases?
• RQ3 . Can quantum superpositions and entanglement lead to undecidability in decision problems that are decidable in the classical case?
• RQ4 . How do quantum control and reasoning under partial observability differ in computability from classical counterparts?
• RQ5 . Are there quantum analogues of classical POMDP algorithms, and what are their complexity bounds?

Key findings

• . Goal-state reachability is decidable for POMDPs, as it can be reduced to finite-state MDP reachability and solved via graph-based analysis of policy-induced state transitions.
• . In contrast, goal-state reachability is undecidable for QOMDPs, proven via reduction from the matrix mortality problem, which is known to be undecidable.
• . The existence of a policy achieving a certain expected reward has the same complexity (PSPACE-complete) in both QOMDPs and POMDPs for finite and infinite horizons.
• . The undecidability arises from the ability of quantum operations to entangle and superpose states in ways that encode undecidable computational problems.
• . The paper establishes that quantum systems can simulate non-deterministic, non-terminating processes that are impossible to analyze algorithmically in finite time.
• . Despite the undecidability of reachability, the paper shows that policy existence remains in PSPACE for both models, indicating a sharp distinction between policy existence and reachability complexity.