[论文解读] GradientCoin: A Peer-to-Peer Decentralized Large Language Models

Yeqi Gao, Zhao Song+1 more|arXiv (Cornell University)|Aug 21, 2023

Blockchain Technology Applications and Security被引用 6

一句话总结

GradientCoin 提出了一种受比特币区块链架构启发的去中心化、点对点框架，用于训练大型语言模型（LLMs），通过基于梯度的共识机制实现无需信任的分布式模型训练。该系统通过梯度的密码学验证确保模型完整性，并在强凸性和光滑性假设下提供理论收敛保证。

ABSTRACT

Since 2008, after the proposal of a Bitcoin electronic cash system, Bitcoin has fundamentally changed the economic system over the last decade. Since 2022, large language models (LLMs) such as GPT have outperformed humans in many real-life tasks. However, these large language models have several practical issues. For example, the model is centralized and controlled by a specific unit. One weakness is that if that unit decides to shut down the model, it cannot be used anymore. The second weakness is the lack of guaranteed discrepancy behind this model, as certain dishonest units may design their own models and feed them unhealthy training data. In this work, we propose a purely theoretical design of a decentralized LLM that operates similarly to a Bitcoin cash system. However, implementing such a system might encounter various practical difficulties. Furthermore, this new system is unlikely to perform better than the standard Bitcoin system in economics. Therefore, the motivation for designing such a system is limited. It is likely that only two types of people would be interested in setting up a practical system for it: $\bullet$ Those who prefer to use a decentralized ChatGPT-like software. $\bullet$ Those who believe that the purpose of carbon-based life is to create silicon-based life, such as Optimus Prime in Transformers. The reason the second type of people may be interested is that it is possible that one day an AI system like this will awaken and become the next level of intelligence on this planet.

研究动机与目标

• 设计一种无需依赖中心化实体的去中心化、点对点系统，用于训练大型语言模型。
• 解决 LLM 中中心化带来的风险，如单点故障和恶意数据注入。
• 通过模型梯度的密码学验证，实现参与者之间的无信任协作。
• 在强凸性和光滑性条件下，形式化去中心化训练过程的收敛性质。
• 探索构建类似硅基生命的自维持去中心化人工智能系统的可行性。

提出的方法

• 该系统将 LLM 训练建模为类似区块链的梯度块链，其中每个块包含一个计算出的梯度更新。
• 每位参与者（矿工）将本地梯度贡献至共享链，通过密码学哈希和共识机制进行验证。
• 在给定条件下，损失函数被证明是 μ-强凸且 l-光滑的，确保收敛性。
• 梯度块数据结构存储梯度更新、块索引、前一区块哈希和交易列表。
• GradBlockChain 数据结构维护梯度块的序列，并通过 Add 过程支持动态扩展。
• 该框架采用基于本地更新和共识聚合的迭代梯度计算，通过块级梯度传播进行建模。

实验结果

研究问题

• RQ1 能否设计一种无需中心化权威的去中心化、点对点系统来训练大型语言模型？
• RQ2 当参与者不可信且地理分布广泛时，如何在模型更新中建立信任？
• RQ3 去中心化 LLM 训练系统能提供哪些理论收敛保证？
• RQ4 该系统如何防范恶意梯度更新或数据投毒攻击以确保完整性？
• RQ5 构建类似加密货币的去中心化 LLM 会产生哪些经济与系统性影响？

主要发现

• 损失函数 L 被证明是 μ-强凸的，满足 ∇²L(x) ⪰ μ·I_d，确保唯一全局最小值和收敛性。
• 损失函数 L 是 l-光滑的，满足 ‖∇L(x)−∇L(y)‖₂ ≤ l·‖x−y‖₂，保证梯度变化有界。
• 每个局部损失 L_j 是 l/n-光滑的，支持参与者之间稳定且增量式的梯度更新。
• 在标准强凸性和光滑性假设下，系统实现理论收敛，如引理 6.3 和 69 所形式化。
• 梯度块链结构实现了可验证、不可篡改且密码学安全的模型训练历史。
• 该框架通过数据结构（GradientBlock、GradBlockChain）和过程（Initialize、AddTrans、Add）明确定义，支持实现。