RoboPapers Ep#3 Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids with Toru Lin
Sim-to-real training is an important part of the future of robot skill learning. But a lot of sim-to-real work focuses on navigation, grasping, or, more recently, non-interactive robot behaviors like dancing. Training dexterous policies for humanoids is very different, because manipulation is a very hard problem and multi-finger dexterous hands are even more difficult.
Enter this cool work from Toru Lin and colleagues. Their goal is to do long-horizon manipulation with dexterous humanoid robots.
Reinforcement learning has delivered promising results in achieving human- or even superhuman-level capabilities across diverse problem domains, but success in dexterous robot manipulation remains limited. This work investigates the key challenges in applying reinforcement learning to solve a collection of contact-rich manipulation tasks on a humanoid embodiment. We introduce novel techniques to overcome the identified challenges with empirical validation. Our main contributions include an automated real-to-sim tuning module that brings the simulated environment closer to the real world, a generalized reward design scheme that simplifies reward engineering for long-horizon contact-rich manipulation tasks, a divide-and-conquer distillation process that improves the sample efficiency of hard-exploration problems while maintaining sim-to-real performance, and a mixture of sparse and dense object representations to bridge the sim-to-real perception gap. We show promising results on three humanoid dexterous manipulation tasks, with ablation studies on each technique. Our work presents a successful approach to learning humanoid dexterous manipulation using sim-to-real reinforcement learning, achieving robust generalization and high performance without the need for human demonstration.
ArXiv
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com