[논문 리뷰] LogicEnvGen: Task-Logic Driven Generation of Diverse Simulated Environments for Embodied AI

Jianan Wang, Siyang Zhang외 5명|arXiv (Cornell University)|2026. 01. 20.

Artificial Intelligence in Games인용 수 0

한 줄 요약

LogicEnvGen은 LLM 주도 프레임워크로, embodied AI를 위한 테스트 케이스로서 논리적으로 다양하고 물리적으로 그럴듯한 시뮬레이션 환경을 생성하고, 작업별 행동 계획을 도출하며, 논리적 궤적을 수집하고, 물리적으로 타당한 환경 배치를 구성합니다. 또한 네 가지 지표를 갖춘 벤치마크인 LogicEnvEval을 도입합니다.

ABSTRACT

Simulated environments play an essential role in embodied AI, functionally analogous to test cases in software engineering. However, existing environment generation methods often emphasize visual realism (e.g., object diversity and layout coherence), overlooking a crucial aspect: logical diversity from the testing perspective. This limits the comprehensive evaluation of agent adaptability and planning robustness in distinct simulated environments. To bridge this gap, we propose LogicEnvGen, a novel method driven by Large Language Models (LLMs) that adopts a top-down paradigm to generate logically diverse simulated environments as test cases for agents. Given an agent task, LogicEnvGen first analyzes its execution logic to construct decision-tree-structured behavior plans and then synthesizes a set of logical trajectories. Subsequently, it adopts a heuristic algorithm to refine the trajectory set, reducing redundant simulation. For each logical trajectory, which represents a potential task situation, LogicEnvGen correspondingly instantiates a concrete environment. Notably, it employs constraint solving for physical plausibility. Furthermore, we introduce LogicEnvEval, a novel benchmark comprising four quantitative metrics for environment evaluation. Experimental results verify the lack of logical diversity in baselines and demonstrate that LogicEnvGen achieves 1.04-2.61x greater diversity, significantly improving the performance in revealing agent faults by 4.00%-68.00%.

연구 동기 및 목표

• logically diverse simulated environments to evaluate agent adaptability and planning robustness.
• Develop a top-down framework that derives behavior plans from task logic and instantiates diverse environments accordingly.
• Ensure physical plausibility through constraint solving during object placement and layout design.
• Provide a benchmark (LogicEnvEval) and metrics to quantify physical plausibility, logical diversity, and fault-revealing efficacy.

제안 방법

• Phase 1: Behavior Plan Derivation uses LLMs to decompose a task into independent subtasks and generate a decision-tree structured behavior plan for each subtask.
• Phase 2: Logical Trajectory Collection synthesizes distinct logical trajectories by combining decision paths from subtasks, using a Minimal Trajectory Selection heuristic to prune redundancy.
• Phase 3: Simulated Environment Construction instantiates concrete environments for each trajectory via floor plan design, object selection from asset libraries, and CSP-based constraint solving to arrange objects physically plausibly.
• Enforce physical plausibility by modeling object placement as a Constraint Satisfaction Problem (CSP) solved with Z3, with a relaxation mechanism for infeasible peripheral relations.
• Introduce LogicEnvEval, a 25-task household benchmark with four agent policies (one correct, three faulty) and four evaluation metrics.
• Employ a constrained trajectory prompt design to guide LLMs and reduce hallucinations during environment generation.

실험 결과

연구 질문

• RQ1 작업 실행 로직을 어떻게 임베디드 에이전트를 위한 다양하고 논리적으로 구분된 테스트 환경으로 변환할 수 있는가?
• RQ2 상향식의 로직 주도 생성 프로세스가 기저 방법과 비교해 논리적 커버리지와 결함 탐지 측면에서 얼마나 개선되는가?
• RQ3 제약 해결 및 궤적 가지치기가 물리적 타당성과 효율성 보장에 어떤 역할을 하는가?
• RQ4 LogicEnvEval가 물리적 타당성, 논리적 다양성, 시나리오 타당성, 결함 탐지 측정에 얼마나 효과적인가?
• RQ5 이 접근법이 서로 다른 LLM에 대해 일반화될 수 있으며 모델 능력의 변화에도 성능을 유지하는가?

주요 결과

• LogicEnvGen은 기본 대비 논리적 커버리지를 1.04–2.61배 더 크게 달성합니다.
• LogicEnvGen은 잘못된 정책에서의 결함 탐지 비율을 기저 대비 4.00%–68.00% 향상시킵니다.
• 제약 기반 객체 배치( CSP 및 Z3 활용 )를 강제하면 물리적 타당성(PhyPR)이 차원별로 현저히 향상됩니다.
• 최소 궤적 선택 알고리즘은 중복을 줄이고 더 낮은 계산 비용으로 높은 커버리지를 유지합니다.
• LogicEnvEval은 25개의 장기 과업과 정확한 정책 및 잘못된 정책 두 유형에 기초한 다차원 벤치마크(PhyPR, LogCov, SceVR, FauDR)를 제공합니다.