[Paper Review] ChatHouseDiffusion: Prompt-Guided Generation and Editing of Floor Plans

Sizhong Qin, Chengyu He+5 more|arXiv (Cornell University)|Oct 15, 2024

Model-Driven Software Engineering Techniques4 citations

TL;DR

ChatHouseDiffusion proposes a novel, interactive floor plan generation and editing framework that integrates large language models (LLMs) for natural language understanding, Graphormer for topological encoding, and diffusion models for high-fidelity layout synthesis. It enables iterative, localized edits via prompt-guided refinement, achieving higher IoU scores and significantly improving design flexibility and efficiency over one-step or non-interactive methods.

ABSTRACT

The generation and editing of floor plans are critical in architectural planning, requiring a high degree of flexibility and efficiency. Existing methods demand extensive input information and lack the capability for interactive adaptation to user modifications. This paper introduces ChatHouseDiffusion, which leverages large language models (LLMs) to interpret natural language input, employs graphormer to encode topological relationships, and uses diffusion models to flexibly generate and edit floor plans. This approach allows iterative design adjustments based on user ideas, significantly enhancing design efficiency. Compared to existing models, ChatHouseDiffusion achieves higher Intersection over Union (IoU) scores, permitting precise, localized adjustments without the need for complete redesigns, thus offering greater practicality. Experiments demonstrate that our model not only strictly adheres to user specifications but also facilitates a more intuitive design process through its interactive capabilities.

Motivation & Objective

• To address the inflexibility of existing floor plan generation methods that require complete input specifications and lack support for iterative, localized edits.
• To improve design efficiency by enabling users to refine layouts through natural language feedback without full redesigns.
• To integrate LLMs for semantic understanding of user intent, Graphormer for structural topology modeling, and diffusion models for high-quality layout generation.
• To support precise, localized edits by modifying attention maps in the diffusion process, enabling targeted changes without re-generating the entire layout.
• To create a practical, user-friendly design pipeline that accommodates evolving, incomplete user inputs through iterative refinement.

Proposed method

• Leverages a large language model (LLM) accessed via OpenAI API to parse natural language prompts into structured JSON-formatted room specifications, including type, size, location, and connectivity.
• Employs Graphormer to encode the topological relationships between rooms using a graph representation of room connections and spatial positions.
• Utilizes a diffusion model to generate high-resolution floor plans (64×64) from the structured input, with denoising guided by both the LLM-structured prompt and graph-based topology.
• Enables localized editing by replacing the attention map in the diffusion model with a modified version that focuses on specific regions, allowing precise modifications without full re-generation.
• Integrates a GUI built with Python's tkinter library to support drawing, text input, seed-based regeneration, and undo functionality for interactive design.
• Uses a prompt engineering strategy that combines user input with a predefined JSON schema to ensure consistent, valid output from the LLM, including enumerated types for room types, locations, and sizes.

Experimental results

Research questions

• RQ1 Can a unified framework combining LLMs, Graphormer, and diffusion models enable flexible, interactive floor plan generation from incomplete or evolving natural language descriptions?
• RQ2 How effectively can localized edits be performed on generated floor plans without full re-generation, and what mechanisms enable such precision?
• RQ3 To what extent does the integration of topological structure (via Graphormer) improve the structural coherence and realism of generated floor plans compared to baseline methods?
• RQ4 How does the proposed method outperform existing one-step or non-interactive generation models in terms of design accuracy and user satisfaction?
• RQ5 Can the system maintain high fidelity and consistency across multiple iterative design steps while adapting to user feedback?

Key findings

• ChatHouseDiffusion achieves higher Intersection over Union (IoU) scores than existing models, indicating superior alignment between generated and ground-truth layouts.
• The model supports precise, localized edits by modifying attention maps in the diffusion process, enabling targeted changes without re-generating the entire floor plan.
• Users can iteratively refine designs using partial or evolving natural language input, significantly enhancing design flexibility and efficiency.
• The system strictly adheres to user-specified constraints, including room types, sizes, locations, and connections, as verified through structured JSON output and validation.
• The integration of LLMs with diffusion models and Graphormer enables a more intuitive and interactive design process, reducing the burden of explicit input specifications.
• The GUI implementation with features like undo, seed control, and prompt memory enhances usability and supports practical, real-time design workflows.