[论文解读] The Representational Alignment Hypothesis: Evidence for and Consequences of Invariant Semantic Structure Across Embedding Modalities

Akhil Ramidi, Kevin Scharp|arXiv (Cornell University)|Feb 18, 2026

Embodied and Extended Cognition被引用 0

一句话总结

该论文回顾独立训练的跨模态嵌入在语义几何上存在不变性证据，并讨论哲学影响，包括拒绝柏拉图式解读并以元语义学为依据将这一想法落地。与此同时强调简单线性映射能够使跨模态嵌入空间对齐。

ABSTRACT

There is growing evidence that independently trained AI systems come to represent the world in the same way. In other words, independently trained embeddings from text, vision, audio, and neural signals share an underlying geometry. We call this the Representational Alignment Hypothesis (RAH) and investigate evidence for and consequences of this claim. The evidence is of two kinds: (i) internal structure comparison techniques, such as representational similarity analysis and topological data analysis, reveal matching relational patterns across modalities without explicit mapping; and (ii) methods based on cross-modal embedding alignment, which learn mappings between representation spaces, show that simple linear transformations can bring different embedding spaces into close correspondence, suggesting near-isomorphism. Taken together, the evidence suggests that, even after controlling for trivial commonalities inherent in standard data preprocessing and embedding procedures, a robust structural correspondence persists, hinting at an underlying organizational principle. Some have argued that this result shows that the shared structure is getting at a fundamental, Platonic level of reality. We argue that this conclusion is unjustified. Moreover, we aim to give the idea an alternative philosophical home, rooted in contemporary metasemantics (i.e., theories of what makes a representation and what makes something meaningful) and responses to the symbol grounding problem. We conclude by considering the scope of the RAH and proposing new ways of distinguishing semantic structures that are genuinely invariant from those that inevitably arise due to the fact that all our data is generated under human-specific conditions on Earth.

研究动机与目标

• 评估在独立训练的嵌入（文本、视觉、听觉、神经信号）中是否存在不变的、独立模态的语义结构。
• 回顾内部结构分析（RSA、拓扑）以及跨模态对齐的证据，显示共享的几何结构且未使用显式映射。
• 评估对符号定位与元语义学的影响，并主张反对柏拉图 Representations 假说。
• 识别普遍不变性面临的挑战并提出未来研究方向。

提出的方法

• 讨论 Representational Similarity Analysis (RSA)、互信息以及拓扑数据分析，用于在各模态内部比较关系模式，而不依赖显式跨模态映射。
• 检查全局几何与拓扑特征，以评估跨模态的共享空间形状。
• 回顾通过线性或近线性映射进行空间对齐的变换型方法（如 Procrustes、CSLS、无监督/弱监督方法）。
• 总结跨文本、跨视觉、跨听觉和神经数据的跨模态对齐证据，显示嵌入空间近似同构。
• 结合符号定位与元语义学文献，解释为何可能出现这样的不变结构。

实验结果

研究问题

• RQ1 在独立训练的跨模态嵌入空间（文本、视觉、听觉、神经数据）中，是否存在共同的不变语义结构？
• RQ2 简单线性变换是否足以对齐这些空间，暗示它们的语义几何接近同构？
• RQ3 不变的嵌入几何对符号定位与元语义学在哲学与实际中的意义为何？
• RQ4 在声明跨模态与环境的普遍不变性方面存在哪些挑战？

主要发现

• 内部结构比较方法揭示跨模态的关系模式相匹配，而未使用显式跨模态映射。
• 变换型方法表明简单的线性映射能够使不同嵌入空间接近对应，暗示近似同构。
• 证据与讨论涵盖神经、文本、视觉和听觉模态，涉及多样任务和数据集。
• 柏拉图 Representations 假说被拒绝或被视为对观察到的对齐无正当理由的解释。
• Representational Alignment 假说被置于元语义学与符号定位框架内，而非柏拉图式现实论。