The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Trends in Reinforcement Learning with Chelsea Finn - #335
Jan 2, 2020
Chelsea Finn, Assistant Professor at Stanford University, shares her insights on advancements in reinforcement learning. She breaks down model-based approaches and the challenges of exploration in complex environments like Montezuma's Revenge. The discussion also touches on the importance of curriculum learning in robotics and the nuances of batch off-policy learning. With exciting implications for real-world applications, Chelsea highlights the evolving landscape of RL libraries and their role in bridging the gap between simulation and practical deployment.
AI Snips
Chapters
Transcript
Episode notes
Dexterous Manipulation and Rubik's Cube
- Two papers demonstrated dexterous manipulation with robotic hands, one using simulation and the other real-world training.
- OpenAI's Rubik's Cube solver sparked controversy due to perceived overhyping, but highlighted the difficulty of physical manipulation.
Model-Based RL
- Model-based RL, which learns a world model for optimization, is gaining traction.
- It offers sample efficiency, but poses challenges in vision-based domains due to pixel prediction.
Batch Off-Policy RL
- Batch off-policy RL learns from fixed datasets without new data collection.
- It's crucial for real-world scenarios where data is limited, costly, or unsafe to acquire.