[Paper Review] Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence

Jiaxing Zhang, Ruyi Gan+23 more|arXiv (Cornell University)|Sep 7, 2022

Scientific Computing and Data Management44 citations

TL;DR

Proposes an open-source Chinese foundation-model ecosystem (Fengshenbang) with three components—Models, Framework, and Benchmark—plus a 49-model catalog and Chinese-specific evaluation benchmarks to enable accessible, resource-efficient Chinese NLP development.

ABSTRACT

Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent challenges: existing foundation models are dominated by the English-language community; users are often given limited resources and thus cannot always use foundation models. To support the development of the Chinese-language community, we introduce an open-source project, called Fengshenbang, which leads by the research center for Cognitive Computing and Natural Language (CCNL). Our project has comprehensive capabilities, including large pre-trained models, user-friendly APIs, benchmarks, datasets, and others. We wrap all these in three sub-projects: the Fengshenbang Model, the Fengshen Framework, and the Fengshen Benchmark. An open-source roadmap, Fengshenbang, aims to re-evaluate the open-source community of Chinese pre-trained large-scale models, prompting the development of the entire Chinese large-scale model community. We also want to build a user-centered open-source ecosystem to allow individuals to access the desired models to match their computing resources. Furthermore, we invite companies, colleges, and research institutions to collaborate with us to build the large-scale open-source model-based ecosystem. We hope that this project will be the foundation of Chinese cognitive intelligence.

Motivation & Objective

• Address the resource and language gap in foundation models dominated by English-language communities.
• Create a comprehensive, user-centered Chinese foundation-model ecosystem integrating models, tooling, and benchmarks.
• Provide open-source governance and collaboration to advance the Chinese large-scale model community.

Proposed method

• Define a User-Centered Taxonomy (UCT) to classify user needs and map to model offerings.
• Assemble and open-source a catalog of 49 Chinese models across NLU, NLG, NLT, and multimodal/domain/exploration tasks (with naming conventions).
• Develop the Fengshen Framework to combine standard data processing, model interfaces, tutorials, docker-like environments, and industry-standard APIs (HuggingFace/Megatron-LM/DeepSpeed integrations).
• Create Fengshenbang Benchmark to enable fair, future-oriented evaluation including a Chinese SuperGLUE-like Chinese leaderboard and knowledge-based QA benchmarks (QAKM).
• Describe model design, selection criteria (power, diversity, usability), and the naming scheme for easy discovery.

Experimental results

Research questions

• RQ1 How can a comprehensive, standardized, and user-centered Chinese foundation-model ecosystem be designed and evaluated?
• RQ2 What model taxonomy, naming, and selection criteria best support Chinese NLP progress and accessibility?
• RQ3 How can tooling and benchmarks enable fair comparisons and ease of use for researchers and practitioners with varying resources?

Key findings

• Introduction of Fengshenbang as a three-part ecosystem: Fengshenbang Model, Fengshen Framework, and Fengshen Benchmark.
• Release and documentation of 49 open Chinese models with a clear naming convention and user-centered taxonomy.
• Creation of a framework that integrates HuggingFace, Megatron-LM, PyTorch-Lightning, and DeepSpeed to enable training and fine-tuning of very large models (over 10B parameters).
• Development of Chinese-centric benchmarks, including plans for Chinese-SuperGLUE and QAKM, to support fair evaluation and progress tracking.
• A practical three-step usage flow: choose a pre-trained Chinese model, refine with Fengshen Framework tutorials, and evaluate on Fengshenbang Benchmarks or custom tasks.
• Emphasis on ethical considerations and ongoing community-driven development to shape the Chinese open-source model ecosystem.