[Paper Review] Missing Information and Asset Allocation

Jean‐Philippe Bouchaud, Marc Potters+1 more|ArXiv.org|Jul 4, 1997

Stock Market Forecasting Methods3 references19 citations

TL;DR

This paper proposes a constrained portfolio optimization framework using generalized entropies to prevent overconcentration on few assets due to incomplete financial data. By introducing a free-utility function akin to thermodynamic free energy, it enforces minimal diversification through constraints on generalized entropy measures, yielding sub-efficient frontiers that align with market practice and reduce instability from noisy covariance estimates.

ABSTRACT

When the available statistical information is imperfect, it is dangerous to follow standard optimisation procedures to construct an optimal portfolio, which usually leads to a strong concentration of the weights on very few assets. We propose a new way, based on generalised entropies, to ensure a minimal degree of diversification.

Motivation & Objective

• To address the instability and overconcentration of optimal portfolios in Markowitz-style mean-variance optimization due to imperfect statistical estimates.
• To formalize the intuitive need for diversification as a constraint on information content, linking portfolio concentration to entropy measures.
• To develop a framework that balances risk-return optimization with minimal diversification, reflecting limited information available to fund managers.
• To generalize the Markowitz efficient frontier by introducing sub-efficient frontiers constrained by effective number of assets or generalized entropy.
• To establish a thermodynamic analogy where a 'free-utility' function combines utility and entropy to model uncertainty in asset allocation.

Proposed method

• Define generalized entropy indicators $ Y_q = \sum_i p_i^q $, with $ Y_2 $ representing the inverse of the effective number of assets $ M_{\text{eff}} = 1/Y_2 $.
• Constrain the optimization to maintain $ M_{\text{eff}} \geq M_0 $, ensuring a minimal level of diversification.
• Modify the Markowitz optimization by adding a Lagrange multiplier $ \nu $ to the diagonal of the covariance matrix $ C_{ij} $, forming $ \mathcal{C}_{ij} = C_{ij} + \nu \delta_{ij} $.
• Derive the optimal weights as $ \mathbf{p} = \frac{1}{2} \mathcal{C}^{-1} [\lambda \mathbf{r} + \mu \mathbf{1}] $, with constraints on return, normalization, and $ M_{\text{eff}} $.
• Introduce a free-utility function $ \mathcal{F}_q = U - \nu \frac{Y_q - 1}{q - 1} $, where $ U $ is the utility (e.g., risk-adjusted return), and $ \nu $ acts as a temperature-like parameter reflecting uncertainty.
• Relate the entropy $ \mathcal{S} = -\sum_i p_i \log p_i $ to $ Y_q $ via $ \mathcal{S} = -\lim_{q \to 1} \frac{Y_q - 1}{q - 1} $, grounding the method in information theory.

Experimental results

Research questions

• RQ1 How can portfolio optimization be made robust to incomplete or noisy statistical estimates of returns and covariances?
• RQ2 What is the information-theoretic basis for requiring portfolio diversification when statistical data are limited?
• RQ3 How can a constrained optimization framework prevent overconcentration on a few assets while maintaining risk-return efficiency?
• RQ4 What is the role of generalized entropies in measuring and limiting the information content of a portfolio allocation?
• RQ5 How does the analogy with thermodynamic free energy help in constructing a rational utility function under uncertainty?

Key findings

• The method generates sub-efficient frontiers that are less concentrated than the standard Markowitz frontier, especially when $ \nu > 0 $, by constraining $ M_{\text{eff}} \geq M_0 $.
• The optimal weights are derived via a modified covariance matrix $ \mathcal{C}_{ij} = C_{ij} + \nu \delta_{ij} $, where $ \nu $ is tuned to satisfy the diversification constraint.
• When $ \nu \to \infty $, the solution converges to equal weights $ p_i = 1/M $, corresponding to maximum entropy and minimal information content.
• The effective number of assets $ M_{\text{eff}} = 1/Y_2 $ provides a quantitative measure of diversification, with higher values indicating broader allocation.
• The free-utility function $ \mathcal{F}_q = U - \nu \frac{Y_q - 1}{q - 1} $ unifies risk-return utility with entropy, where $ \nu $ controls the trade-off between performance and robustness.
• The framework is generalizable beyond variance-based risk to other risk measures, such as Value-at-Risk, making it applicable in non-Gaussian return environments.