[Paper Review] Complexity measure, kernel density estimation, bandwidth selection, and the efficient market hypothesis

Matthieu Garcin|arXiv (Cornell University)|May 22, 2023

Complex Systems and Time Series Analysis4 citations

TL;DR

This paper proposes a novel bandwidth selection method for kernel density estimation in financial time series by maximizing a new complexity measure derived from information theory and complex systems. The method avoids overfitting and underfitting, and applied to Bitcoin data, it shows that conclusions about market efficiency under the efficient market hypothesis (EMH) critically depend on bandwidth choice, highlighting the need for robust, complexity-aware estimation methods in finance.

ABSTRACT

We are interested in the nonparametric estimation of the probability density of price returns, using the kernel approach. The output of the method heavily relies on the selection of a bandwidth parameter. Many selection methods have been proposed in the statistical literature. We put forward an alternative selection method based on a criterion coming from information theory and from the physics of complex systems: the bandwidth to be selected maximizes a new measure of complexity, with the aim of avoiding both overfitting and underfitting. We review existing methods of bandwidth selection and show that they lead to contradictory conclusions regarding the complexity of the probability distribution of price returns. This has also some striking consequences in the evaluation of the relevance of the efficient market hypothesis. We apply these methods to real financial data, focusing on the Bitcoin.

Motivation & Objective

• To address the critical dependence of market efficiency assessments on bandwidth selection in kernel density estimation.
• To develop a complexity-based bandwidth selection method that avoids overfitting and underfitting in nonparametric density estimation of financial returns.
• To reconcile the measurement of market complexity with the evaluation of the efficient market hypothesis (EMH) using information-theoretic tools.
• To demonstrate that conflicting conclusions about EMH consistency arise from different bandwidth selection methods, even with the same data.
• To advocate for multi-method evaluation of market efficiency, emphasizing the role of complexity-aware estimation in financial econophysics.

Proposed method

• Proposes a new complexity measure for kernel density estimates based on divergence from uniformity and Shannon entropy, inspired by the LMC complexity framework.
• Introduces a bandwidth selection criterion that maximizes this complexity measure to achieve optimal smoothing.
• Applies the method to daily Bitcoin (BTC-USD) price returns using kernel density estimation with variable bandwidths.
• Compares the proposed method with classical bandwidth selection techniques (e.g., Silverman’s rule of thumb, plug-in methods) via simulation.
• Employs market information—a measure based on entropy differences between conditional and marginal return distributions—to evaluate EMH consistency.
• Uses the Hurst exponent as a complementary efficiency indicator, contrasting its bandwidth-independent nature with bandwidth-sensitive complexity and market information measures.

Figure 1: Top left and right and bottom left: true (dotted line), and estimated densities, with $h_{c}$ , $h_{\text{AMISE}}$ , $h_{\text{PIT}}$ , $h_{\text{lik}}$ (from the darkest to the lightest), for the Gaussian, Gaussian mixture, and Student simulations. Bottom right: complexity with respect to

Experimental results

Research questions

• RQ1 How does the choice of bandwidth in kernel density estimation affect the assessment of market efficiency via the efficient market hypothesis (EMH)?
• RQ2 Can a complexity-based criterion improve bandwidth selection in nonparametric density estimation of financial returns?
• RQ3 To what extent do different bandwidth selection methods lead to contradictory conclusions about the EMH for the same financial asset, such as Bitcoin?
• RQ4 How does the proposed complexity-maximizing bandwidth selection method compare to classical methods in terms of robustness and consistency with market efficiency?
• RQ5 What is the role of information-theoretic complexity measures in reconciling the analysis of financial market complexity and market efficiency?

Key findings

• The proposed complexity-maximizing bandwidth selection method avoids overfitting and underfitting, providing a robust alternative to classical bandwidth selection techniques.
• For Bitcoin, market information values suggest inefficiency in 2017 and 2020, particularly when small bandwidths are used, indicating potential for statistical arbitrage.
• Larger bandwidths lead to decreasing market information, falsely suggesting EMH consistency, demonstrating the risk of bandwidth-induced bias.
• The Hurst exponent for Bitcoin is above 0.5 in 2017 and 2018 (0.528 and 0.537), indicating long-range dependence, which contradicts the EMH's assumption of random walks.
• Empirical results show that conclusions about EMH consistency vary significantly depending on bandwidth selection, underscoring the need for methodologically cautious, multi-method evaluation.
• The study concludes that market efficiency analysis must integrate complexity-aware estimation, as no single method or statistic is sufficient to determine EMH validity.

Figure 2: Complexity $\mathcal{C}_{h}$ of the annual pdf of daily price returns for BTC-USD, estimated with a kernel approach, with respect to the bandwidth $h$ between 0 and the specific $h_{p}$ calculated for each density. The left graph is between 2015 and 2018 (from the lightest to the darkest c