[论文解读] Simplicity bias, algorithmic probability, and the random logistic map

Boumediene Hamzi, Kamaludin Dingle|arXiv (Cornell University)|Jan 1, 2023

Statistical Mechanics and Entropy被引用 1

一句话总结

本文通过算法概率与信息论，研究了随机逻辑斯蒂映射——一种随机动力系统——中的简洁性偏差。研究发现，简洁性偏差在加性噪声下依然存在，但随噪声增加而减弱，并揭示了预测置信度中的反直觉效应，为噪声复杂系统中的时间序列分析提供了一个统一的概率-复杂性框架。

ABSTRACT

Simplicity bias is an intriguing phenomenon prevalent in various input-output maps, characterized by a preference for simpler, more regular, or symmetric outputs. Notably, these maps typically feature high-probability outputs with simple patterns, whereas complex patterns are exponentially less probable. This bias has been extensively examined and attributed to principles derived from algorithmic information theory and algorithmic probability. In a significant advancement, it has been demonstrated that the renowned logistic map and other one-dimensional maps exhibit simplicity bias when conceptualized as input-output systems. Building upon this work, our research delves into the manifestations of simplicity bias within the random logistic map, specifically focusing on scenarios involving additive noise. We discover that simplicity bias is observable in the random logistic map for specific ranges of $μ$ and noise magnitudes. Additionally, we find that this bias persists even with the introduction of small measurement noise, though it diminishes as noise levels increase. Our studies also revisit the phenomenon of noise-induced chaos, particularly when $μ=3.83$, revealing its characteristics through complexity-probability plots. Intriguingly, we employ the logistic map to illustrate a paradoxical aspect of data analysis: more data adhering to a consistent trend can occasionally lead to \emph{reduced} confidence in extrapolation predictions, challenging conventional wisdom. We propose that adopting a probability-complexity perspective in analyzing dynamical systems could significantly enrich statistical learning theories related to series prediction and analysis. This approach not only facilitates a deeper understanding of simplicity bias and its implications but also paves the way for novel methodologies in forecasting complex systems behavior.

研究动机与目标

• 将简洁性偏差——对简单、对称输出的偏好——的理解从确定性映射扩展至随机的、类现实世界系统。
• 研究加性噪声如何影响逻辑斯蒂映射中的简洁性偏差，特别是噪声诱导混沌区域的情况。
• 通过揭示算法概率下更一致数据反而降低外推预测置信度的反直觉现象，挑战数据解析中的传统直觉。
• 提出一个概率-复杂性框架，统一动力系统分析与算法信息论及机器学习。
• 为利用算法概率原理改进复杂、不确定且噪声时间序列的预测与分析，奠定理论基础。

提出的方法

• 将随机逻辑斯蒂映射建模为带加性噪声的输入-输出系统，采用标准形式 x_{k+1} = μx_k(1 - x_k) + η_k。
• 应用算法概率与柯尔莫哥洛夫复杂度，基于简洁性偏差边界估算观测输出模式的概率。
• 生成复杂度-概率图，可视化不同参数区域下模式复杂度与输出概率之间的反比关系。
• 系统性地改变 μ（分岔参数）与噪声强度，评估简洁性偏差的鲁棒性与衰减特性。
• 通过数值模拟分析 μ = 3.83 时的噪声诱导混沌，将其与复杂度及概率分布的变化相联系。
• 比较不同一致性水平数据集的预测置信度，识别基于算法概率推断中的悖论行为。

实验结果

研究问题

• RQ1 在加性测量噪声下，简洁性偏差是否仍存在于随机逻辑斯蒂映射中？
• RQ2 增加噪声强度如何影响系统中简洁性偏差的强度与可检测性？
• RQ3 μ = 3.83 时的噪声诱导混沌现象能否通过复杂度-概率图表征？
• RQ4 为何在算法概率下，更一致的数据趋势有时反而导致外推预测置信度降低？
• RQ5 概率-复杂性框架在多大程度上能统一动力系统分析并提升基于机器学习的时间序列预测？

主要发现

• 在分岔参数 μ 的特定范围及低至中等噪声水平下，可在随机逻辑斯蒂映射中观测到简洁性偏差。
• 随着噪声强度增加，简洁性偏差的强度减弱，尤其在噪声诱导混沌区域更为显著。
• 在 μ = 3.83 时，噪声诱导混沌表现为复杂度-概率分布的显著偏移，通过算法分析确认了其存在。
• 在算法概率评估下，更一致的数据趋势可能反而降低外推预测的置信度，挑战经典统计直觉。
• 研究证实，算法概率为理解多样动力系统中的简洁性偏差提供统一视角，即使在随机扰动下亦成立。
• 研究结果支持将复杂度-概率分析作为改进机器学习模型在复杂、不确定系统中预测能力的稳健理论基础。