[Paper Review] Statistical Measures of Complexity: Why?

David P. Feldman, James P. Crutchfield|ArXiv.org|Aug 22, 1997

Complex Systems and Decision Making4 references16 citations

TL;DR

This paper critically evaluates the LMC statistical complexity measure $C_{\rm LMC}$, showing it fails to be extensive and becomes trivial when modified to be so. It argues that a useful complexity measure must not only vanish in ordered and disordered limits but also have a clear physical interpretation of what structural features it quantifies.

ABSTRACT

We review several statistical complexity measures proposed over the last decade and a half as general indicators of structure or correlation. Recently, Lopez-Ruiz, Mancini, and Calbet [Phys. Lett. A 209 (1995) 321] introduced another measure of statistical complexity C_{LMC} that, like others, satisfies the ``boundary conditions'' of vanishing in the extreme ordered and disordered limits. We examine some properties of C_{LMC} and find that it is neither an intensive nor an extensive thermodynamic variable and that it vanishes exponentially in the thermodynamic limit for all one-dimensional finite-range spin systems. We propose a simple alteration of C_{LMC} that renders it extensive. However, this remedy results in a quantity that is a trivial function of the entropy density and hence of no use as a measure of structure or memory. We conclude by suggesting that a useful ``statistical complexity'' must not only obey the ordered-random boundary conditions of vanishing, it must also be defined in a setting that gives a clear interpretation to what structures are quantified.

Motivation & Objective

• To assess the validity and utility of the LMC statistical complexity measure $C_{\rm LMC}$ as a general indicator of structural complexity.
• To investigate whether $C_{\rm LMC}$ satisfies thermodynamic scaling properties, particularly extensivity in the thermodynamic limit.
• To determine whether modifications to $C_{\rm LMC}$ that restore extensivity result in a meaningful measure of structure or reduce it to a trivial function of entropy.
• To argue that boundary conditions alone (vanishing in ordered and disordered limits) are insufficient to define a useful statistical complexity measure.
• To emphasize the need for a clear physical or informational interpretation of what structural features a complexity measure quantifies.

Proposed method

• Analyzes the LMC complexity measure defined as $C_{\rm LMC}[Y] = H[Y] \cdot D[Y]$, where $H[Y]$ is Shannon entropy and $D[Y]$ is disequilibrium.
• Examines the behavior of $C_{\rm LMC}$ in one-dimensional finite-range spin systems under the thermodynamic limit.
• Proposes a modified version of $C_{\rm LMC}$ to achieve extensivity by rescaling the disequilibrium term.
• Demonstrates that the modified measure becomes a trivial function of the entropy density, losing its ability to distinguish structural patterns.
• Uses analytical and thermodynamic arguments to show that $C_{\rm LMC}$ vanishes exponentially in the thermodynamic limit for such systems.
• Compares $C_{\rm LMC}$ to other complexity measures like excess entropy and logical depth to highlight interpretational clarity.

Experimental results

Research questions

• RQ1 Does the LMC complexity measure $C_{\rm LMC}$ exhibit extensive scaling in the thermodynamic limit for one-dimensional spin systems?
• RQ2 Can a modified version of $C_{\rm LMC}$ be made extensive without losing its sensitivity to structural correlations?
• RQ3 Why is the vanishing of complexity in ordered and disordered limits insufficient to define a useful statistical complexity measure?
• RQ4 What criteria are necessary for a statistical complexity measure to be physically meaningful and interpretable?
• RQ5 How do different complexity measures (e.g., excess entropy, logical depth) compare in terms of interpretability and utility?

Key findings

• The LMC complexity measure $C_{\rm LMC}$ vanishes exponentially in the thermodynamic limit for all one-dimensional finite-range spin systems.
• The LMC measure is neither intensive nor extensive, violating a key requirement for a thermodynamically meaningful complexity measure.
• A proposed modification to make $C_{\rm LMC}$ extensive results in a quantity that is a trivial function of the entropy density alone.
• The modified measure loses all sensitivity to structural correlations and thus cannot serve as a useful indicator of pattern or memory.
• The boundary condition of vanishing in ordered and disordered limits is necessary but not sufficient for a useful complexity measure.
• A meaningful statistical complexity measure must have a clear physical or informational interpretation of what structural features it quantifies.