Rustacean Station

Polars with Ritchie Vink

Jan 5, 2024
Guest
Ritchie Vink
Ritchie Vink, Creator of Polars, discusses data frames, Polars vs Pandas, using Polars in app development, and the 1.0 release of Polars. They also talk about the challenges of growing Polars without bloat and the changes and challenges in the Rust programming language.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
INSIGHT

Polars & Arrow

  • Polars leverages Apache Arrow's in-memory format but implements its own compute engine.
  • This offers potential performance advantages over Pandas, which relies on NumPy and has different memory handling.
INSIGHT

NumPy's Limitations

  • NumPy's limitations with missing data and strings influenced Pandas' design, leading to performance issues.
  • Pandas uses workarounds like casting integers to floats or heap-allocating strings, affecting speed.
INSIGHT

Polars Design Philosophy

  • Polars prioritized a database-like query engine design, unlike Pandas.
  • This allows for query optimization and efficient handling of DataFrame operations.
Get the Snipd Podcast app to discover more snips from this episode
Get the app