Meta Tech Podcast 78: Generating 3D Worlds with AI
Sep 19, 2025
Mahima and Rakesh dive into the challenges of 3D content creation, revealing how AssetGen aims to make it accessible for all. They discuss the intricacies of 3D design and the role of large language models in solving these challenges. The duo also explores the fascinating concept of generating complete 3D worlds from simple prompts. Plus, they share insights on the future of 3D perception and generation, promising even richer experiences in the metaverse.
AI Snips
Chapters
Transcript
Episode notes
GenAI In Horizon Desktop Tools
- Horizon provides a Desktop Unity Editor where creators can build worlds, characters, and animations.
- The editor already includes GenAI features that generate 3D assets from text prompts.
AssetGen's Multi‑Stage Pipeline
- AssetGen is a pipeline of models and non‑ML stages that separately generate geometry and textures.
- They pretrain on noisy 3D data then fine‑tune on curated quality datasets to overcome scarcity.
Toward Unified 3D Diffusion
- Unifying shape and appearance into a single 3D diffusion model could reduce reprojection artifacts.
- The team actively researches joint diffusion over geometry and color to improve fidelity.