[Paper Review] Understanding understanding: a renormalization group inspired model of (artificial) intelligence

Antal Jakovác, D. Berenyi+1 more|arXiv (Cornell University)|Oct 26, 2020

Cognitive Science and Education Research1 references4 citations

TL;DR

This paper proposes a renormalization group-inspired framework that defines understanding as the reorganization of information through a new coordinate system, distinguishing relevant (deterministic) from irrelevant (uniformly distributed) parameters. It formalizes scientific and artificial intelligence understanding via a measure of relevance, enabling lossy compression and clarifying the distinction between pattern recognition and true comprehension.

ABSTRACT

This paper is about the meaning of understanding in scientific and in artificial intelligent systems. We give a mathematical definition of the understanding, where, contrary to the common wisdom, we define the probability space on the input set, and we treat the transformation made by an intelligent actor not as a loss of information, but instead a reorganization of the information in the framework of a new coordinate system. We introduce, following the ideas of physical renormalization group, the notions of relevant and irrelevant parameters, and discuss, how the different AI tasks can be interpreted along these concepts, and how the process of learning can be described. We show, how scientific understanding fits into this framework, and demonstrate, what is the difference between a scientific task and pattern recognition. We also introduce a measure of relevance, which is useful for performing lossy compression.

Motivation & Objective

• To formalize the concept of understanding in both artificial intelligence and natural sciences using a mathematical framework grounded in information reorganization.
• To address the limitations of traditional scientific and AI methods in explaining complex systems where individual parameters lose interpretability.
• To introduce a measure of relevance that enables lossy data compression while preserving predictive power.
• To clarify the distinction between pattern recognition and genuine scientific understanding through the lens of coordinate system transformation.
• To demonstrate how learning in time-dependent systems can be structured using the relevance measure and renormalization group principles.

Proposed method

• Define understanding as a transformation of input data into a new coordinate system, where information is reorganized rather than lost.
• Introduce the concepts of relevant and irrelevant parameters: relevant coordinates have deterministic values over a subset, while irrelevant ones are uniformly distributed.
• Use a parameter k to represent measurement precision or spatial/temporal averaging, enabling the definition of relevance through the behavior of coordinates across k-scale variations.
• Apply renormalization group running to track how coordinates transition from relevant to irrelevant as scale changes, identifying critical points of relevance.
• Construct a measure of relevance based on the k-value at which a coordinate becomes statistically irrelevant, allowing for lossy compression.
• Extend the framework to multiple disjoint subsets, enabling application to classification, regression, and decoding tasks via inspection of relevant/irrelevant coordinates.

Experimental results

Research questions

• RQ1 How can understanding in artificial intelligence and natural sciences be formally defined beyond mere pattern recognition?
• RQ2 What mathematical criteria distinguish relevant from irrelevant parameters in a system's description?
• RQ3 How can a measure of relevance be constructed to enable effective lossy compression of complex data?
• RQ4 In what way does the renormalization group framework help resolve the interpretability crisis in deep learning and complex simulations?
• RQ5 How does the framework distinguish between scientific understanding and superficial correlation in data-driven models?

Key findings

• Understanding is redefined as the selection of a coordinate system where relevant parameters are deterministic and irrelevant ones are uniformly distributed over their domain.
• The framework enables lossy compression by identifying coordinates that become irrelevant at a certain scale, with the measure of relevance derived from k-dependent measurements.
• The distinction between scientific understanding and pattern recognition is formalized: scientific understanding involves interpretable, relevant effects that can be named and studied individually.
• In systems with many relevant and irrelevant coordinates, individual parameters lose meaning, and only collective behavior is descriptive—mirroring the behavior of deep neural networks.
• The model explains why some predictions fail: apparent randomness may stem from omitting hidden, yet relevant, parameters that were misclassified as irrelevant.
• The framework supports all standard AI tasks (classification, regression, compression, decoding) through direct inspection of relevant and irrelevant coordinates.