GOTO - The Brightest Minds in Tech This AI Fights Health Insurance Denials • Holden Karau & Julian Wood
7 snips
Jan 23, 2026 Holden Karau, an open-source engineer and Apache Spark contributor, shares her journey in tech and her mission with Fight Health Insurance, an AI tool designed to help users appeal health insurance claim denials. She delves into the technical intricacies of distributed data processing, discussing Spark's performance advantages over MapReduce. Holden unveils the challenges of building the appeals assistant, from gathering patient data to navigating insurance regulations, all while emphasizing the significance of sustainable funding for open-source projects.
AI Snips
Chapters
Transcript
Episode notes
Spark As The Conductor Of Big Data
- Apache Spark acts as a conductor that coordinates many tools to process data that does not fit on a single machine.
- Arrow became the lingua franca to move data between JVM-based systems and GPU-native ML libraries.
ML Training Breaks Traditional Fault Models
- Machine learning workflows change failure handling: you cannot simply recompute a lost worker's work during distributed training.
- Gang scheduling and new resource-aware scheduling features are required to handle ML across many machines and GPUs.
Request GPUs Only When Needed
- Use resource profiles to request GPUs only for stages that need them rather than reserving them for entire jobs.
- Tag critical parts of workflows so Spark schedules expensive hardware efficiently and lowers total cost.