[Paper Review] Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Framework

Changyu Du, Sebastian Esser+2 more|arXiv (Cornell University)|Aug 15, 2024

BIM and Construction Integration6 citations

TL;DR

Text2BIM leverages a multi-agent LLM framework to convert natural language into executable code that generates editable BIM models in Vectorworks, with rule-based quality checks and iterative refinement.

ABSTRACT

The conventional BIM authoring process typically requires designers to master complex and tedious modeling commands in order to materialize their design intentions within BIM authoring tools. This additional cognitive burden complicates the design process and hinders the adoption of BIM and model-based design in the AEC (Architecture, Engineering, and Construction) industry. To facilitate the expression of design intentions more intuitively, we propose Text2BIM, an LLM-based multi-agent framework that can generate 3D building models from natural language instructions. This framework orchestrates multiple LLM agents to collaborate and reason, transforming textual user input into imperative code that invokes the BIM authoring tool's APIs, thereby generating editable BIM models with internal layouts, external envelopes, and semantic information directly in the software. Furthermore, a rule-based model checker is introduced into the agentic workflow, utilizing predefined domain knowledge to guide the LLM agents in resolving issues within the generated models and iteratively improving model quality. Extensive experiments were conducted to compare and analyze the performance of three different LLMs under the proposed framework. The evaluation results demonstrate that our approach can effectively generate high-quality, structurally rational building models that are aligned with the abstract concepts specified by user input. Finally, an interactive software prototype was developed to integrate the framework into the BIM authoring software Vectorworks, showcasing the potential of modeling by chatting. The code is available at: https://github.com/dcy0577/Text2BIM

Motivation & Objective

• Reduce the cognitive burden of BIM authoring by enabling natural-language to BIM model generation.
• Enable collaborative reasoning among specialized LLM agents to produce structured, executable BIM code.
• Integrate domain knowledge through a rule-based model checker to guide iterative model quality improvements.
• Demonstrate a working prototype that integrates Text2BIM with Vectorworks for interactive modeling.

Proposed method

• Four specialized LLM agents (Product Owner, Architect, Programmer, Reviewer) collaborate to convert natural language into imperative BIM API calls.
• A toolset of 26 high-level BIM functions encapsulates Vectorworks API calls and domain rules, guiding code generation.
• Architect prompts produce building plans using architectural rules; Product Owner enriches instructions and coordinates with Architect via function calling.
• Programmer writes Python code that calls only the defined tool functions and standard Python libraries; a custom interpreter executes and tests code.
• A triple-loop workflow combines code generation, self-reflection, and model quality assessment with a rule-based model checker and BCF reporting to iteratively fix issues.
• Memory modules (local/global) store interaction history to maintain context across iterations.

Experimental results

Research questions

• RQ1 Can an LLM-based multi-agent system generate coherent, executable BIM code from natural language instructions?
• RQ2 How can architectural rules and domain knowledge be integrated to improve model quality and ensure consistency with BIM standards?
• RQ3 What is the role of a rule-based model checker and BCF-driven feedback in guiding iterative improvements to BIM models?
• RQ4 How effective is a Vectorworks-based prototype in showcasing modeling-by-chatting for early-design BIM generation?

Key findings

• The framework enables generation of editable BIM models with external envelopes, internal layouts, and semantic information from natural language input.
• A multi-agent collaboration with a rule-based checker can guide iterative improvements to model quality through several feedback loops.
• Experiments compare three LLMs within the Text2BIM framework to assess performance in producing structurally rational and semantically enriched models.
• An interactive prototype demonstrates integration with Vectorworks, illustrating practical modeling-by-chat workflows.