[Paper Review] Dynamic programming approach to principal-agent problems

Jakša Cvitanić, Dylan Possamaï+1 more|arXiv (Cornell University)|Oct 24, 2015

Stochastic processes and financial applications53 references111 citations

TL;DR

This paper develops a dynamic programming approach to solve general continuous-time Principal-Agent problems with terminal lump-sum contracts, even when the agent controls both drift and volatility of output. By restricting contracts to those enabling a dynamic programming representation of the agent’s value function, the authors reduce the non-zero-sum stochastic game to a solvable stochastic control problem, proving this restriction incurs no loss of optimality. The key contribution is a systematic method applicable to non-Markovian, volatility-controlling settings using second-order backward SDEs.

ABSTRACT

We consider a general formulation of the Principal-Agent problem with a lump-sum payment on a finite horizon, providing a systematic method for solving such problems. Our approach is the following: we first find the contract that is optimal among those for which the agent's value process allows a dynamic programming representation, for which the agent's optimal effort is straightforward to find. We then show that the optimization over the restricted family of contracts represents no loss of generality. As a consequence, we have reduced this non-zero sum stochastic differential game to a stochastic control problem which may be addressed by the standard tools of control theory. Our proofs rely on the backward stochastic differential equations approach to non-Markovian stochastic control, and more specifically, on the recent extensions to the second order case.

Motivation & Objective

• To address the general Principal-Agent problem in continuous-time with lump-sum payments, especially when the agent controls both drift and volatility of output.
• To overcome the difficulty of solving non-Markovian, non-zero-sum stochastic control problems in moral hazard settings.
• To provide a systematic method that reduces the complex contract design problem to a standard stochastic control framework.
• To show that restricting to contracts allowing a dynamic programming representation of the agent’s value function entails no loss of optimality.

Proposed method

• Introduce a restricted family of contracts for which the agent’s value process admits a dynamic programming representation.
• Use second-order backward stochastic differential equations (2BSDEs) to characterize the agent’s value process in non-Markovian settings.
• Apply the Hamilton-Jacobi-Bellman (HJB) equation framework to the agent’s optimization problem under this restricted class.
• Derive first-order conditions to identify the agent’s optimal effort as the maximizer of the Hamiltonian.
• Prove that the supremum of the principal’s expected utility over this restricted class equals the global supremum, ensuring no loss of optimality.
• Leverage recent results on 2BSDEs and regularity bypass techniques to handle non-dominated measures arising from volatility control.

Experimental results

Research questions

• RQ1 Can a systematic method be developed to solve Principal-Agent problems in continuous-time when the agent controls both drift and volatility of output?
• RQ2 Does restricting to contracts that allow a dynamic programming representation of the agent’s value function lead to a loss of optimality in the contract design problem?
• RQ3 How can the principal’s optimization problem be reduced to a standard stochastic control problem in non-Markovian, volatility-controlling settings?
• RQ4 What role do the first- and second-order sensitivities (to the output and its quadratic variation) play in characterizing optimal contracts?
• RQ5 Can the approach be extended to general utility functions and multi-dimensional output processes without Markovian assumptions?

Key findings

• The optimal contract is characterized by solving a stochastic control problem derived from the HJB equation, where the agent’s optimal effort is determined by maximizing the Hamiltonian.
• The method applies to general utility functions and multi-dimensional models where the agent controls both drift and volatility, extending beyond previous CARA and linear models.
• The restriction to contracts allowing a dynamic programming representation of the agent’s value function incurs no loss of optimality under mild technical conditions.
• The agent’s value process is represented via second-order backward SDEs, enabling the analysis of non-Markovian, volatility-controlling problems.
• For the special case of CARA preferences and linear output dynamics, the optimal contract depends on both the terminal output and its quadratic variation, generalizing earlier linear contracts.
• The approach recovers known results such as the first-best risk-sharing contract in the absence of moral hazard, showing that the optimal contract can replicate first-best outcomes through incentive alignment.