The Good Fight David Bau on How Artificial Intelligence Works
85 snips
Sep 30, 2025 David Bau, an Assistant Professor and expert on deep generative networks, joins Yascha Mounk to dive into the complexities of AI. They discuss the critical need for understanding behind AI technologies and the implications of many not grasping their workings. Bau clarifies the distinction between generative models and classifiers, explaining how neural networks are constructed. He highlights the transformative role of transformers and shared insights on training methods, emphasizing the importance of moral alignment in AI development.
AI Snips
Chapters
Transcript
Episode notes
Analyze Models After Training
- Do post-hoc interpretability: analyze trained networks to understand emergent computations.
- Treat networks like complex biological systems and dissect their learned structures after training.
Next-Word Prediction Is Core
- Language modeling reduces generation to repeated next-word classification over large vocabularies.
- Scale of inputs and outputs (big vocab and long contexts) is key to LLM capabilities.
Transformers Add Attention Memory
- The transformer architecture introduced attention, a short-term memory mechanism called attention.
- Attention lets models use recent inputs as memory and reason over them efficiently.