Gradient Dissent: Conversations on AI Why Physical AI Needed a Completely New Data Stack
101 snips
Dec 16, 2025 Nikolaus West, CEO and co-founder of Rerun, specializes in revolutionizing how we log and visualize multimodal sensor data for robotics. He discusses the challenges of taking AI from the lab to the real world and how Rerun's innovative data stack solves these issues. Key topics include breakthrough techniques in manipulation, the intriguing blend of reinforcement and imitation learning, and the significance of open-source tools in advancing robotics. Nikolaus also shares insights on the future of consumer robotics and the importance of robust data pipelines.
AI Snips
Chapters
Transcript
Episode notes
Physical Data Needs New Storage Primitives
- Physical data is multimodal, multi-rate and episodic, so tabular formats don't fit well.
- Rerun built a custom Arrow-based format akin to a sparse Parquet to index and query these recordings efficiently.
Visualization Finds Long-Buried Bugs
- Open-source users debug far more and find long-standing bugs by visualizing intermediate pipeline steps.
- Teams discovered training-data bugs that had persisted for years once they inspected recordings at scale.
Real-Data Users Drive Database Need
- Customers who care about real-world performance need to save and query real data, not just simulate.
- Heavy reliance on simulation reduces the need for storage and querying, biasing tools toward live-data users.