The Data Exchange with Ben Lorica Making Data Engineering Safe for Automation and Agents
12 snips
Nov 13, 2025 Ciro Greco, Co-founder and CEO of Bauplan, discusses revolutionizing data engineering by applying software principles like version control and transactional pipelines to data lakes. He highlights the unique challenges of data work, such as scale and fragmentation, and introduces a git-like branching model for enhanced reproducibility. Ciro emphasizes the importance of transactional guarantees, especially for automated agents, and advocates for a code-first approach to enable safe and efficient interactions with data platforms.
AI Snips
Chapters
Transcript
Episode notes
Enable Team Autonomy With Branches
- Give product and analytics teams their own cloud branches to develop without central-team bottlenecks.
- Monitor and roll back those branches so teams stay independent and safe.
Aim For Transactional Pipelines
- Atomic table operations aren't enough; pipelines often touch many tables and need multi-table transactional guarantees.
- Make pipeline runs atomic so the lake never becomes incoherent mid-run.
Agents Create Database‑Style Concurrency
- Agents will massively increase concurrent reads and writes to lakes, creating database-like concurrency challenges.
- The only scalable solution is transactional behavior and isolation in the lake.