Workshop: Deep Reinforcement Learning
Pieter Abbeel, Chelsea Finn, Joelle Pineau, David Silver, Satinder Singh, Coline Devin, Misha Laskin, Kimin Lee, Janarthanan Rajendran, Vivek Veeriah
2020-12-11T08:30:00-08:00 - 2020-12-11T19:00:00-08:00
Abstract: In recent years, the use of deep neural networks as function approximators has enabled researchers to extend reinforcement learning techniques to solve increasingly complex control tasks. The emerging field of deep reinforcement learning has led to remarkable empirical results in rich and varied domains like robotics, strategy games, and multiagent interactions. This workshop will bring together researchers working at the intersection of deep learning and reinforcement learning, and it will help interested researchers outside of the field gain a high-level view about the current state of the art and potential directions for future contributions.
Video
Chat
Chat is not available.
Schedule
2020-12-11T08:29:00-08:00 - 2020-12-11T08:30:00-08:00
Welcome and Introduction
2020-12-11T08:30:00-08:00 - 2020-12-11T09:00:00-08:00
Invited talk: PierreYves Oudeyer "Machines that invent their own problems: Towards open-ended learning of skills"
Pierre-Yves Oudeyer
2020-12-11T09:00:00-08:00 - 2020-12-11T09:15:00-08:00
Contributed Talk: Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning
Sammy Christen, Lukas Jendele, Emre Aksan, Otmar Hilliges
2020-12-11T09:15:00-08:00 - 2020-12-11T09:30:00-08:00
Contributed Talk: Maximum Reward Formulation In Reinforcement Learning
Sai Krishna Gottipati, Yashaswi Pathak, Rohan Nuttall, Sahir ., Ravi Chunduru, Ahmed Touati, Sriram Ganapathi, Matthew Taylor , Sarath Chandar
2020-12-11T09:30:00-08:00 - 2020-12-11T09:45:00-08:00
Contributed Talk: Accelerating Reinforcement Learning with Learned Skill Priors
Karl Pertsch, Youngwoon Lee, Joseph Lim
2020-12-11T09:45:00-08:00 - 2020-12-11T10:00:00-08:00
Contributed Talk: Asymmetric self-play for automatic goal discovery in robotic manipulation
OpenAI Robotics, Matthias Plappert, Raul Sampedro, Tao Xu , Ilge Akkaya, Vineet Kosaraju, Peter Welinder, Ruben D'Sa, Arthur Petron, Henrique Ponde, Alex Paino, Hyeonwoo Noh Noh , Lilian Weng, Qiming Yuan, Casey Chu , Wojciech Zaremba
2020-12-11T10:00:00-08:00 - 2020-12-11T10:30:00-08:00
Invited talk: Marc Bellemare "Autonomous navigation of stratospheric balloons using reinforcement learning"
Marc Bellemare
2020-12-11T10:30:00-08:00 - 2020-12-11T11:00:00-08:00
Break
2020-12-11T10:59:00-08:00 - 2020-12-11T11:00:00-08:00
Introduction
2020-12-11T11:00:00-08:00 - 2020-12-11T11:30:00-08:00
Invited talk: Peter Stone "Grounded Simulation Learning for Sim2Real with Connections to Off-Policy Reinforcement Learning"
Peter Stone
For autonomous robots to operate in the open, dynamically changing world, they will need to be able to learn a robust set of skills from relatively little experience. This talk introduces Grounded Simulation Learning as a way to bridge the so-called reality gap between simulators and the real world in order to enable transfer learning from simulation to a real robot. Grounded Simulation Learning has led to the fastest known stable walk on a widely used humanoid robot. Connections to theoretical advances in off-policy reinforcement learning will be highlighted.
2020-12-11T11:30:00-08:00 - 2020-12-11T11:45:00-08:00
Contributed Talk: Mirror Descent Policy Optimization
Manan Tomar, Lior Shani, Yonathan Efroni, Mohammad Ghavamzadeh
2020-12-11T11:45:00-08:00 - 2020-12-11T12:00:00-08:00
Contributed Talk: Planning from Pixels using Inverse Dynamics Models
Keiran Paster, Sheila McIlraith, Jimmy Ba
2020-12-11T12:00:00-08:00 - 2020-12-11T12:30:00-08:00
Invited talk: Matt Botvinick "Alchemy: A Benchmark Task Distribution for Meta-Reinforcement Learning Research"
Matt Botvinick
2020-12-11T12:30:00-08:00 - 2020-12-11T13:30:00-08:00
Poster session 1
2020-12-11T13:29:00-08:00 - 2020-12-11T13:30:00-08:00
Introduction
2020-12-11T13:30:00-08:00 - 2020-12-11T14:00:00-08:00
Invited talk: Susan Murphy "We used RL but…. Did it work?!"
Susan Murphy
Digital Healthcare is a growing area of importance in modern healthcare due to its potential in helping individuals improve their behaviors so as to better manage chronic health challenges such as hypertension, mental health, cancer and so on. Digital apps and wearables, observe the user's state via sensors/self-report, deliver treatment actions (reminders, motivational messages, suggestions, social outreach,...) and observe rewards repeatedly on the user across time. This area is seeing increasing interest by RL researchers with the goal of including in the digital app/wearable an RL algorithm that "personalizes" the treatments to the user. But after RL is run on a number of users, how do we know whether the RL algorithm actually personalized the sequential treatments to the user? In this talk we report on our first efforts to address this question after our RL algorithm was deployed on each of 111 individuals with hypertension.
2020-12-11T14:00:00-08:00 - 2020-12-11T14:15:00-08:00
Contributed Talk: MaxEnt RL and Robust Control
Benjamin Eysenbach, Sergey Levine
2020-12-11T14:15:00-08:00 - 2020-12-11T14:30:00-08:00
Contributed Talk: Reset-Free Lifelong Learning with Skill-Space Planning
Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch
2020-12-11T14:30:00-08:00 - 2020-12-11T15:00:00-08:00
Invited talk: Anusha Nagabandi "Model-based Deep Reinforcement Learning for Robotic Systems"
Anusha Nagabandi
Deep learning has shown promising results in robotics, but we are still far from having intelligent systems that can operate in the unstructured settings of the real world, where disturbances, variations, and unobserved factors lead to a dynamic environment. In this talk, we'll see that model-based deep RL can indeed allow for efficient skill acquisition, as well as the ability to repurpose models to solve a variety of tasks. We'll scale up these approaches to enable locomotion with a 6-DoF legged robot on varying terrains in the real world, as well as dexterous manipulation with a 24-DoF anthropomorphic hand in the real world. We then focus on the inevitable mismatch between an agent's training conditions and the test conditions in which it may actually be deployed, thus illuminating the need for adaptive systems. Inspired by the ability of humans and animals to adapt quickly in the face of unexpected changes, we present a meta-learning algorithm within this model-based RL framework to enable online adaptation of large, high-capacity models using only small amounts of data from the new task. These fast adaptation capabilities are seen in both simulation and the real-world, with experiments such as a 6-legged robot adapting online to an unexpected payload or suddenly losing a leg. We will then further extend the capabilities of our robotic systems by enabling the agents to reason directly from raw image observations. Bridging the benefits of representation learning techniques with the adaptation capabilities of meta-RL, we'll present a unified framework for effective meta-RL from images. With robotic arms in the real world that learn peg insertion and ethernet cable insertion to varying targets, we'll see the fast acquisition of new skills, directly from raw image observations in the real world. Finally, this talk will conclude that model-based deep RL provides a framework for making sense of the world, thus allowing for reasoning and adaptation capabilities that are necessary for successful operation in the dynamic settings of the real world.
2020-12-11T15:00:00-08:00 - 2020-12-11T15:30:00-08:00
Break
2020-12-11T15:29:00-08:00 - 2020-12-11T15:30:00-08:00
Introduction
2020-12-11T15:30:00-08:00 - 2020-12-11T16:00:00-08:00
Invited talk: Ashley Edwards "Learning Offline from Observation"
Ashley Edwards
A common trope in sci-fi is to have a robot that can quickly solve some problem after watching a person, studying a video, or reading a book. While these settings are (currently) fictional, the benefits are real. Agents that can solve tasks by observing others have the potential to greatly reduce the burden of their human teachers, removing some of the need to hand-specify rewards or goals. In this talk, I consider the question of how an agent can not only learn by observing others, but also how it can learn quickly by training offline before taking any steps in the environment. First, I will describe an approach that trains a latent policy directly from state observations, which can then be quickly mapped to real actions in the agent’s environment. Then I will describe how we can train a novel value function, Q(s,s’), to learn off-policy from observations. Unlike previous imitation from observation approaches, this formulation goes beyond simply imitating and rather enables learning from potentially suboptimal observations.
2020-12-11T16:00:00-08:00 - 2020-12-11T16:07:00-08:00
NeurIPS RL Competitions: Flatland challenge
Sharada Mohanty
2020-12-11T16:07:00-08:00 - 2020-12-11T16:15:00-08:00
NeurIPS RL Competitions: Learning to run a power network
Antoine Marot
2020-12-11T16:15:00-08:00 - 2020-12-11T16:22:00-08:00
NeurIPS RL Competitions: Procgen challenge
Sharada Mohanty
2020-12-11T16:22:00-08:00 - 2020-12-11T16:30:00-08:00
NeurIPS RL Competitions: MineRL
William Guss, Stephanie Milani
2020-12-11T16:30:00-08:00 - 2020-12-11T17:00:00-08:00
Invited talk: Karen Liu "Deep Reinforcement Learning for Physical Human-Robot Interaction"
Karen Liu
Creating realistic virtual humans has traditionally been considered a research problem in Computer Animation primarily for entertainment applications. With the recent breakthrough in collaborative robots and deep reinforcement learning, accurately modeling human movements and behaviors has become a common challenge also faced by researchers in robotics and artificial intelligence. For example, mobile robots and autonomous vehicles can benefit from training in environments populated with ambulating humans and learning to avoid colliding with them. Healthcare robotics, on the other hand, need to embrace physical contacts and learn to utilize them for enabling human’s activities of daily living. An immediate concern in developing such an autonomous and powered robotic device is the safety of human users during the early development phase when the control policies are still largely suboptimal. Learning from physically simulated humans and environments presents a promising alternative which enables robots to safely make and learn from mistakes without putting real people at risk. However, deploying such policies to interact with people in the real world adds additional complexity to the already challenging sim-to-real transfer problem. In this talk, I will present our current progress on solving the problem of sim-to-real transfer with humans in the environment, actively interacting with the robots through physical contacts. We tackle the problem from two fronts: developing more relevant human models to facilitate robot learning and developing human-aware robot perception and control policies. As an example of contextualizing our research effort, we develop a mobile manipulator to put clothes on people with physical impairments, enabling them to carry out day-to-day tasks and maintain independence.
2020-12-11T17:00:00-08:00 - 2020-12-11T18:00:00-08:00
Panel discussion
Pierre-Yves Oudeyer, Marc Bellemare, Peter Stone, Matt Botvinick, Susan Murphy, Anusha Nagabandi, Ashley Edwards, Karen Liu, Pieter Abbeel
2020-12-11T18:00:00-08:00 - 2020-12-11T19:00:00-08:00