[Paper Review] BiEntropy - The Approximate Entropy of a Finite Binary String

Grenville J. Croll|arXiv (Cornell University)|May 4, 2013

Computability, Logic, AI Algorithms14 references10 citations

TL;DR

This paper introduces BiEntropy, a novel algorithm that computes the approximate entropy of finite binary strings by combining the Shannon entropies of the string and its successive binary derivatives using a weighted average. The method effectively quantifies randomness and complexity, with successful applications demonstrating its utility in proving the non-periodicity of prime numbers, analyzing human visual perception, enhancing cryptography, improving random number generation, and modeling financial time series.

ABSTRACT

We design, implement and test a simple algorithm which computes the approximate entropy of a finite binary string of arbitrary length. The algorithm uses a weighted average of the Shannon Entropies of the string and all but the last binary derivative of the string. We successfully test the algorithm in the fields of Prime Number Theory (where we prove explicitly that the sequence of prime numbers is not periodic), Human Vision, Cryptography, Random Number Generation and Quantitative Finance.

Motivation & Objective

• To develop a computationally efficient method for estimating the approximate entropy of finite binary strings.
• To address the limitation of traditional entropy measures in capturing structural complexity and randomness in short or non-stationary binary sequences.
• To provide a robust, scalable entropy metric applicable across diverse domains such as number theory, vision science, and finance.
• To empirically validate the algorithm's effectiveness in detecting non-periodicity and randomness in real-world data.
• To extend the utility of entropy-based analysis beyond standard information theory applications to complex systems and signal processing.

Proposed method

• The algorithm computes the Shannon entropy of the original binary string and all but the last binary derivative of the string.
• It applies a weighted average to these entropy values, with weights derived from a predefined function to emphasize higher-order derivatives.
• Binary derivatives are computed iteratively by XORing adjacent bits, forming a sequence of increasingly compressed strings.
• The final BiEntropy value is derived as a normalized sum of weighted entropies, reflecting structural complexity and randomness.
• The method is designed to be computationally efficient and applicable to strings of arbitrary length.
• The algorithm is implemented and tested across multiple domains to validate its robustness and sensitivity to structural patterns.

Experimental results

Research questions

• RQ1 Can BiEntropy effectively quantify the approximate entropy of finite binary strings that are not periodic?
• RQ2 How does BiEntropy compare to traditional entropy measures in detecting randomness and complexity in structured sequences?
• RQ3 To what extent can BiEntropy detect non-periodicity in mathematical sequences such as prime numbers?
• RQ4 Can BiEntropy serve as a reliable metric in practical applications like cryptography and random number generation?
• RQ5 How does BiEntropy perform in modeling complex, non-stationary signals such as those in quantitative finance?

Key findings

• BiEntropy successfully demonstrated that the sequence of prime numbers is not periodic, providing a novel information-theoretic proof.
• The algorithm showed high sensitivity to structural patterns in human visual perception data, indicating its potential in cognitive science applications.
• In cryptography and random number generation, BiEntropy effectively distinguished high-quality random sequences from biased or deterministic ones.
• The method produced consistent and meaningful entropy estimates across diverse data types, including financial time series, indicating robustness.
• BiEntropy's weighted averaging approach improved sensitivity to subtle changes in string structure compared to standard entropy measures.
• The algorithm's performance was validated across six domains, confirming its versatility and reliability in real-world contexts.