[论文解读] MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
tldr: MeshXL 将三维网格生成公式化为使用 Neural Coordinate Field (NeurCF) 的自回归序列建模,从而实现直接、高保真度的三维网格生成,并作为下游任务的基础模型。随着模型规模和数据规模的扩大,性能与多样性得到改善。
The polygon mesh representation of 3D data exhibits great flexibility, fast rendering speed, and storage efficiency, which is widely preferred in various applications. However, given its unstructured graph representation, the direct generation of high-fidelity 3D meshes is challenging. Fortunately, with a pre-defined ordering strategy, 3D meshes can be represented as sequences, and the generation process can be seamlessly treated as an auto-regressive problem. In this paper, we validate the Neural Coordinate Field (NeurCF), an explicit coordinate representation with implicit neural embeddings, is a simple-yet-effective representation for large-scale sequential mesh modeling. After that, we present MeshXL, a family of generative pre-trained auto-regressive models, which addresses the process of 3D mesh generation with modern large language model approaches. Extensive experiments show that MeshXL is able to generate high-quality 3D meshes, and can also serve as foundation models for various down-stream applications.
研究动机与目标
- 验证 Neural Coordinate Field 作为一种简单而有效的网格表示,用于大规模序列建模。
- 开发 MeshXL 系列生成式预训练变换器,用于直接的三维网格生成,无需中间表示。
- 展示大规模预训练和更大模型规模能够提升网格生成的质量和多样性。
- 展示 X-to-mesh 生成(文本/图像条件)及作为三维基础模型的潜力。
提出的方法
- 通过受 PolyGen 和 MeshGPT 启发的预定义顶点/面排序策略,将 3D 网格表示为序列。
- 在 NeurCF 中用可学习的坐标嵌入对离散化坐标进行编码,从而实现具有显式空间表示的隐式嵌入。
- 在 ShapeNet、3D-FUTURE、Objaverse 和 Objaverse-XL 的 2.5M+ 网格上训练仅解码器的变换器(MeshXL),进行下一个令牌预测。
- 通过为端到端的 MeshXL 训练初始化新的坐标和位置嵌入,从预训练的 OPT 权重中进行微调。
- 通过通过 Q-Former 前缀把图像/文本特征置于前导,来实现带条件的 X-to-mesh 生成。

实验结果
研究问题
- RQ1Can NeurCF provide an effective, scalable coordinate-based representation for direct 3D mesh generation?
- RQ2Does auto-regressive MeshXL pre-training on large-scale 3D mesh data yield higher-quality meshes than prior methods?
- RQ3To what extent do model size and data scale improve mesh generation quality and diversity?
- RQ4Can MeshXL support image- or text-conditioned 3D mesh generation (X-to-mesh) and downstream tasks such as texturing?
- RQ5Is MeshXL capable of serving as a 3D foundation model for various downstream applications?
主要发现
- NeurCF provides a simple, end-to-end coordinate-based representation for unstructured 3D meshes that supports autoregressive generation.
- MeshXL with larger parameter counts and more pre-training data achieves better validation perplexity and improves mesh quality and diversity.
- MeshXL surpasses prior art on ShapeNet categories in unconditional mesh generation (higher diversity and quality metrics).
- MeshXL enables X-to-mesh generation using image or text prompts with a VLM-derived prefix, enabling conditional 3D mesh creation.
- Human studies show MeshXL-generated meshes are preferred over baselines in geometry and design alignment.
- MeshXL demonstrates effective texture generation compatibility with existing texturing pipelines.

阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。