80,000 Hours Podcast #23 - How to actually become an AI alignment researcher, according to Dr Jan Leike
17 snips
Mar 16, 2018 Dr. Jan Leike, a Research Scientist at DeepMind, shares valuable insights on how to join the world's leading AI team. He discusses the importance of completing a computer science and mathematics degree, publishing papers, finding a supportive supervisor, and attending top conferences. Jan also talks about the qualities of a good fit for research and highlights the pressing issue of AGI safety. They also touch upon misconceptions about AI, DeepMind's research focus, and failures of current AI systems.
AI Snips
Chapters
Transcript
Episode notes
Neural Nets Are Overconfident On Unfamiliar Inputs
- Neural networks lack reliable uncertainty estimates and remain overconfident on out-of-distribution inputs.
- That overconfidence yields confident but nonsensical reward assignments rather than admitting ignorance when facing new situations.
Try Anomaly Detection For Distributional Awareness
- Try adding anomaly detection or other mechanisms so agents know when their inputs lie outside training distribution.
- Jan suggests anomaly detection as a candidate defense but notes scalability and effectiveness remain unclear and untested here.
Safety Research Has Many Low Hanging Fruits
- Many safety questions remain underexplored: safe exploration, side-effects regularization, and robust maximization.
- Jan frames these as low-hanging fruit where new researchers can make impactful contributions at present.