[Paper Review] Style-compatible Object Recommendation for Multi-room Indoor Scene Synthesis

Yu He, Yun Cai+6 more|arXiv (Cornell University)|Mar 9, 2020

Remote Sensing and LiDAR Applications25 references4 citations

TL;DR

This paper proposes a style-compatible, multi-room indoor scene synthesis pipeline that uses a Labeled LDA model for category-level object recommendation and combinatorial optimization for instance-level selection, incorporating function diversity and style harmony across rooms. It achieves scene quality comparable to professional designers in user studies, marking the first integration of residence-level style compatibility in indoor scene synthesis.

ABSTRACT

Traditional indoor scene synthesis methods often take a two-step approach: object selection and object arrangement. Current state-of-the-art object selection approaches are based on convolutional neural networks (CNNs) and can produce realistic scenes for a single room. However, they cannot be directly extended to synthesize style-compatible scenes for multiple rooms with different functions. To address this issue, we treat the object selection problem as combinatorial optimization based on a Labeled LDA (L-LDA) model. We first calculate occurrence probability distribution of object categories according to a topic model, and then sample objects from each category considering their function diversity along with style compatibility, while regarding not only separate rooms, but also associations among rooms. User study shows that our method outperforms the baselines by incorporating multi-function and multi-room settings with style constraints, and sometimes even produces plausible scenes comparable to those produced by professional designers.

Motivation & Objective

• Address the limitation of existing indoor scene synthesis methods that focus only on single rooms, leading to function collapse and poor style consistency.
• Enable multi-functional room support where a single room can serve multiple purposes, such as a living room doubling as a dining area.
• Introduce residence-level style compatibility by ensuring consistent material and color palettes across multiple rooms based on a user-provided style reference.
• Develop a unified object recommendation pipeline that jointly optimizes for function diversity, spatial constraints, and style harmony in multi-room settings.
• Demonstrate that automated scene synthesis can produce results comparable to professional interior designers in terms of aesthetics and coherence.

Proposed method

• Model each room as a multi-labeled document and objects as words, using Labeled Latent Dirichlet Allocation (L-LDA) to learn category-level object occurrence probabilities from the SUNCG dataset.
• Generate a room-specific style palette from a user-provided reference image using a 5-color palette extraction method to guide style compatibility.
• Formulate instance-level object recommendation as a combinatorial optimization problem with three key constraints: space occupation, group-level material consistency (e.g., bed and nightstand), and global style harmony across rooms.
• Introduce a cost function with three energy terms: occupation constraint (to avoid overcrowding), group constraint (to enforce material consistency within object groups), and global style constraint (to align with the reference image’s style).
• Use an existing object arrangement method (Anonymous, 2020) after instance selection to generate final, well-arranged 3D scenes.
• Apply a modified version of Chen et al. (2015)’s material suggestion method to improve style compatibility, replacing binary material constraints with group-based consistency modeling.

Experimental results

Research questions

• RQ1 Can a topic model like L-LDA effectively support multi-room, multi-functional indoor scene synthesis by modeling object category distributions across labeled rooms?
• RQ2 How can style compatibility be enforced not only within a room but also across multiple rooms in a residence using a single reference image?
• RQ3 To what extent does incorporating group-level material constraints improve the realism and harmony of synthesized indoor scenes compared to per-object material suggestions?
• RQ4 Can a combinatorial optimization framework that jointly considers function diversity, spatial constraints, and style compatibility produce scenes comparable to those designed by professionals?
• RQ5 How does the proposed method perform in comparison to single-room, single-label baselines in terms of visual quality, style coherence, and functional diversity?

Key findings

• The proposed method significantly outperforms baseline methods in style compatibility and visual aesthetics, with user study scores averaging 4.5/5.0 for the synthesized scenes compared to 3.2/5.0 for original SUNCG scenes.
• The inclusion of the group constraint term reduced material inconsistency between associated objects (e.g., TV stand and coffee table), resulting in more harmonious and realistic room compositions.
• The global style constraint term successfully aligned the style of multiple rooms with the reference image, producing a cohesive residence-wide aesthetic that was rated significantly higher than methods without it.
• The occupation constraint prevented overcrowding, ensuring that rooms remained visually organized and functional, as evidenced by the reduction in chaotic layouts when this term was active.
• User studies with 25 participants, including 5 professional designers, confirmed that scenes generated by the pipeline were rated higher than original SUNCG scenes in all five evaluated aspects: object diversity, consistency, style compatibility, function sufficiency, and visual aesthetics.
• The method successfully mitigates function collapse by allowing multi-labeled rooms, enabling diverse object sets even in functionally overlapping spaces like combined living-dining areas.