The Joe Reis Show DuckLake w/ Hannes Mühleisen - Practical Data Lunch and Learn. June 4, 2025
16 snips
Jun 4, 2025 Discover the innovative Duck Lake extension of DuckDB, designed to revolutionize data management. The episode highlights impressive architecture aimed at enhancing database performance and interoperability. Enjoy a live demo showcasing unique features like effortless data manipulation and intriguing time travel functionalities. Hannes Mühleisen answers questions, making complex topics accessible and engaging!
AI Snips
Chapters
Transcript
Episode notes
Demo: DuckLake with Postgres and S3
- Hannes demoed DuckLake with metadata stored in Postgres and data files on S3.
- The system handled commits, updates, and queries transparently with remote storage and centralized metadata.
DuckLake's Concurrency Solutions
- DuckLake achieves concurrency with short metadata transactions and conflict detection using change logs.
- It improves on older systems by minimizing commit time and managing conflicts efficiently.
DuckLake Encryption Security Model
- Encryption keys for DuckLake files are stored in the metadata server with row-level access control.
- This secures files in untrusted blob storage by requiring metadata access to decrypt data.