[Paper Review] Dynamic Large Language Models on Blockchains

Yuanhao Gong|arXiv (Cornell University)|Jul 20, 2023

Topic Modeling4 citations

TL;DR

This paper proposes training and deploying dynamic large language models (DLLMs) on blockchains to enable continuous, decentralized, and secure model updates. By leveraging blockchain’s decentralization, transparency, and incentive mechanisms, the system allows users to contribute data and developers to iteratively improve models, creating a self-sustaining ecosystem where both parties are economically rewarded based on value contribution.

ABSTRACT

Training and deploying the large language models requires a large mount of computational resource because the language models contain billions of parameters and the text has thousands of tokens. Another problem is that the large language models are static. They are fixed after the training process. To tackle these issues, in this paper, we propose to train and deploy the dynamic large language model on blockchains, which have high computation performance and are distributed across a network of computers. A blockchain is a secure, decentralized, and transparent system that allows for the creation of a tamper-proof ledger for transactions without the need for intermediaries. The dynamic large language models can continuously learn from the user input after the training process. Our method provides a new way to develop the large language models and also sheds a light on the next generation artificial intelligence systems.

Motivation & Objective

• Address the high computational cost and static nature of traditional large language models (LLMs) that cannot adapt after training.
• Overcome centralization risks in LLMs, such as data privacy loss and user access control by corporations.
• Enable continuous learning from user inputs while preserving data privacy through blockchain encryption.
• Establish an economic incentive model where users are rewarded for valuable input and developers earn from high-quality models.
• Create a transparent, tamper-proof, and decentralized system for LLM training and deployment using blockchain infrastructure.

Proposed method

• Deploy dynamic large language models (DLLMs) on blockchain networks to support runtime adaptation and parameter updates.
• Use blockchain’s distributed ledger to store model weights, user inputs, and training logs in a cryptographically secure and immutable manner.
• Implement end-to-end encryption for user-provided data to ensure privacy and prevent unauthorized access.
• Integrate token-based economic incentives: users earn tokens for high-quality inputs, developers earn for deploying effective models.
• Leverage smart contracts to automate model updates, reward distribution, and access control based on contribution value.
• Design a decentralized model registry where only valuable, actively used models and inputs persist, ensuring system efficiency.

Experimental results

Research questions

• RQ1 How can large language models be made dynamically updatable after initial training without relying on centralized control?
• RQ2 What architectural and cryptographic mechanisms enable secure, private, and decentralized model training on blockchains?
• RQ3 How can economic incentives be structured to align user contributions with model quality and developer rewards?
• RQ4 In what ways does blockchain decentralization improve data privacy and system resilience compared to traditional LLM deployment?
• RQ5 Can a self-sustaining ecosystem emerge where both users and developers are economically motivated to contribute to model evolution?

Key findings

• The proposed DLLM-on-blockchain system enables continuous model updates through user input, transforming static LLMs into adaptive systems.
• User data is encrypted and stored securely across the blockchain network, minimizing exposure and ensuring privacy.
• A token-based incentive mechanism allows users to earn rewards for high-quality inputs, while developers earn from model usage.
• The system ensures only valuable inputs and models remain active, creating a self-regulating ecosystem that filters out low-quality contributions.
• Decentralization reduces reliance on single entities, mitigating risks of data loss, censorship, and model monopolization.
• The integration of blockchain provides transparency and immutability, enhancing trust in model training and deployment processes.