The Information Bottleneck

Yaroslav Bulatov, an AI researcher and early member of OpenAI and Google Brain, now pushes to rebuild learning algorithms for far greater energy efficiency. He discusses why current deep learning is wasteful, the idea of replaying AI history with hindsight, hierarchical message passing alternatives, the Muon optimizer breakthrough, and why small, open teams and non-experts can drive rapid innovation.

Kyunghyun Cho, NYU professor of Health Statistics and Computer Science and former Genentech executive, discusses why healthcare is uniquely hard for AI. He explores patient-controlled records, a provocative continuous randomized trial idea, the need for end-to-end drug discovery, mysteries around GLP-1s, antibiotic economics, and how unified language models could compress decades of drug development.

Stefano Ermon, Stanford professor and co-founder/CEO of Inception AI, co-inventor of DDIM and diffusion methods. He explains what diffusion LLMs are and why iterative refinement could overtake autoregressive models. The conversation covers discrete diffusion for text, inference speed and parallel generation, Mercury II’s latency wins, and implications for architectures, tooling, and scaling.

Naomi Saphra, Kempner Research Fellow at Harvard and incoming assistant professor at Boston University, studies training dynamics and interpretability in deep learning. She explains why training is more like evolution than human learning. Topics include grokking and hidden phase transitions, symmetry breaking and head specialization, how code and tokenization shape behavior, and why run-to-run non-determinism matters.

Stefano Soatto, VP for AI at AWS and UCLA professor leading work on agentic AI, discusses treating LLMs as stochastic dynamical systems that require control. He explains strands coding: skeletons with verifiable pre/post-conditions to constrain AI functions. Conversation covers vibe vs spec coding limits, why algorithmic information matters, and how world models emerge from rich multimodal reasoning engines.

Tanishq Abraham, CEO and co-founder of Sophont.ai, builds multimodal foundation models for pathology, neuroimaging, and clinical text. He discusses training on high-quality public data that rivals massive private sets. Conversation covers finding signals doctors can’t see, fusing strong single-modality encoders into multimodal systems, regulatory paths, and practical near-term impacts like pharma partnerships.

Anastasios Angelopoulos, co-founder and CEO of Arena AI and theoretical statistician, explains why static benchmarks fail and how large-scale human-preference leaderboards work. He discusses style control vs substance, measuring AI-generated "slop," tool-use and code evaluation, and how real-user testing and rigorous statistics shape model leaderboards and pre-release testing.

Fred Sala, Assistant Professor at UW–Madison and Chief Scientist at Snorkel AI, works on data-centric AI and weak supervision. He discusses why personalization is the next frontier for LLMs. Short takes cover security risks from personal agents, why prompting fails at scale, activation-steering like REFT as an efficient personalization path, self-distillation for continual learning, and why high-quality data still beats fancy architecture.

Bayan Bruss, VP of Applied AI at a major consumer bank building AI for autonomous financial decision-making. He explores why money is a uniquely hard ML problem. They discuss perception-belief-action frameworks for finance. They debate foundation models versus purpose-built encoders, why synthetic time-series data helps, limits of explainability, and hybrid latent vs language reasoning for financial systems.

David Mezzetti, creator of TextAI and solo developer of an open-source AI orchestration library focused on local-first and small-model workflows. He discusses why local-first AI matters, how COVID research led to semantic search, the power of tiny models on CPU, evolving RAG and orchestration, and the trade-offs of resisting then embracing cloud APIs.