Workshop
3rd Offline Reinforcement Learning Workshop: Offline RL as a "Launchpad"
Aviral Kumar · Rishabh Agarwal · Aravind Rajeswaran · Wenxuan Zhou · George Tucker · Doina Precup · Aviral Kumar
Room 291 - 292
Fri 2 Dec, 6:20 a.m. PST
While offline RL focuses on learning solely from fixed datasets, one of the main learning points from the previous edition of offline RL workshop was that large-scale RL applications typically want to use offline RL as part of a bigger system as opposed to being the end-goal in itself. Thus, we propose to shift the focus from algorithm design and offline RL applications to how offline RL can be a launchpad , i.e., a tool or a starting point, for solving challenges in sequential decision-making such as exploration, generalization, transfer, safety, and adaptation. Particularly, we are interested in studying and discussing methods for learning expressive models, policies, skills and value functions from data that can help us make progress towards efficiently tackling these challenges, which are otherwise often intractable.
Submission site: https://openreview.net/group?id=NeurIPS.cc/2022/Workshop/Offline_RL. The submission deadline is September 25, 2022 (Anywhere on Earth). Please refer to the submission page for more details.
Schedule
Fri 6:20 a.m. - 6:30 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
SlidesLive Video |
🔗 |
Fri 6:30 a.m. - 7:00 a.m.
|
Offline RL in the context of "Collect and Infer" (Martin Riedmiller)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Fri 7:00 a.m. - 7:10 a.m.
|
Efficient Planning in a Compact Latent Action Space
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 7:10 a.m. - 7:20 a.m.
|
Control Graph as Unified IO for Morphology-Task Generalization
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 7:20 a.m. - 7:30 a.m.
|
Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 7:35 a.m. - 8:05 a.m.
|
AV2.0: Learning to Drive at a Global Scale (Alex Kendall)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Fri 8:05 a.m. - 9:10 a.m.
|
Poster Session 1
(
Poster Session
)
>
|
🔗 |
Fri 9:10 a.m. - 9:40 a.m.
|
Learning from Suboptimal Demonstrations with No Rewards (Dorsa Sadigh)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Fri 9:40 a.m. - 10:30 a.m.
|
Break
|
🔗 |
Fri 10:45 a.m. - 11:30 a.m.
|
Panel Discussion 1 - Applications
(
Panel Discussion
)
>
SlidesLive Video |
🔗 |
Fri 11:30 a.m. - 11:40 a.m.
|
Choreographer: Learning and Adapting Skills in Imagination
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 11:40 a.m. - 11:50 a.m.
|
Provable Benefits of Representational Transfer in Reinforcement Learning
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 11:50 a.m. - 12:00 p.m.
|
Pareto-Efficient Decision Agents for Offline Multi-Objective Reinforcement Learning
(
Contributed Talk
)
>
SlidesLive Video |
🔗 |
Fri 12:00 p.m. - 1:00 p.m.
|
Poster Session 2
(
Poster Session
)
>
|
🔗 |
Fri 1:00 p.m. - 1:30 p.m.
|
Reinforcement Learning and LTV at Spotify (Tony Jebara)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Fri 1:30 p.m. - 2:00 p.m.
|
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient (Wen Sun)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Fri 2:00 p.m. - 3:00 p.m.
|
Panel Discussion 2 - Research
(
Panel Discussion
)
>
SlidesLive Video |
🔗 |
Fri 3:00 p.m. - 3:30 p.m.
|
Identification of Dead-ends in Safety-Critical Offline RL (Talyor Killian)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
-
|
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information ( Poster ) > link |
11 presentersRiashat Islam · Manan Tomar · Alex Lamb · Hongyu Zang · Yonathan Efroni · Dipendra Misra · Aniket Didolkar · Xin Li · Harm Van Seijen · Remi Tachet des Combes · John Langford |
-
|
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks ( Poster ) > link | Jesse Farebrother · Joshua Greaves · Rishabh Agarwal · Charline Le Lan · Ross Goroshin · Pablo Samuel Castro · Marc Bellemare 🔗 |
-
|
Confidence-Conditioned Value Functions for Offline Reinforcement Learning ( Poster ) > link | Joey Hong · Aviral Kumar · Sergey Levine 🔗 |
-
|
Efficient Deep Reinforcement Learning Requires Regulating Statistical Overfitting ( Poster ) > link | Qiyang Li · Aviral Kumar · Ilya Kostrikov · Sergey Levine 🔗 |
-
|
Domain Generalization for Robust Model-Based Offline RL
(
Poster
)
>
link
SlidesLive Video |
Alan Clark · Shoaib Siddiqui · Robert Kirk · Usman Anwar · Stephen Chung · David Krueger 🔗 |
-
|
Squeezing more value out of your historical data: data-augmented behavioural cloning as launchpad for reinforcement learning
(
Poster
)
>
link
SlidesLive Video |
Charles Hepburn · Giovanni Montana 🔗 |
-
|
Keep Calm and Carry Offline: Policy refinement in offline reinforcement learning
(
Poster
)
>
link
SlidesLive Video |
Alex Beeson · Giovanni Montana 🔗 |
-
|
Guiding Offline Reinforcement Learning Using a Safety Expert ( Poster ) > link | Richa Verma · Kartik Bharadwaj · Harshad Khadilkar · Balaraman Ravindran 🔗 |
-
|
Pareto-Efficient Decision Agents for Offline Multi-Objective Reinforcement Learning
(
Poster
)
>
link
SlidesLive Video |
Baiting Zhu · Meihua Dang · Aditya Grover 🔗 |
-
|
Revisiting Bellman Errors for Offline Model Selection ( Poster ) > link | Joshua Zitovsky · Rishabh Agarwal · Daniel de Marchi · Michael Kosorok 🔗 |
-
|
Boosting Offline Reinforcement Learning via Data Resampling ( Poster ) > link | Yang Yue · Bingyi Kang · Xiao Ma · Zhongwen Xu · Gao Huang · Shuicheng Yan 🔗 |
-
|
General policy mapping: online continual reinforcement learning inspired on the insect brain
(
Poster
)
>
link
SlidesLive Video |
Angel Yanguas-Gil · Sandeep Madireddy 🔗 |
-
|
Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
(
Poster
)
>
link
SlidesLive Video |
Jiachen Li · Edwin Zhang · Ming Yin · Qinxun Bai · Yu-Xiang Wang · William Yang Wang 🔗 |
-
|
On- and Offline Multi-agent Reinforcement Learning for Disease Mitigation using Human Mobility Data ( Poster ) > link | Sofia Hurtado · Radu Marculescu 🔗 |
-
|
Contrastive Example-Based Control ( Poster ) > link | Kyle Hatch · Sarthak J Shetty · Benjamin Eysenbach · Tianhe Yu · Rafael Rafailov · Russ Salakhutdinov · Sergey Levine · Chelsea Finn 🔗 |
-
|
Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Data ( Poster ) > link | Sunil Madhow · Dan Qiao · Yu-Xiang Wang 🔗 |
-
|
Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
(
Poster
)
>
link
SlidesLive Video |
Shivakanth Sujit · Pedro Braga · Jörg Bornschein · Samira Ebrahimi Kahou 🔗 |
-
|
Offline Policy Comparison with Confidence: Benchmarks and Baselines ( Poster ) > link | Anurag Koul · Mariano Phielipp · Alan Fern 🔗 |
-
|
Residual Model-Based Reinforcement Learning for Physical Dynamics
(
Poster
)
>
link
SlidesLive Video |
Zakariae EL ASRI · Clément Rambour · Vincent LE GUEN · Nicolas THOME 🔗 |
-
|
Raisin: Residual Algorithms for Versatile Offline Reinforcement Learning ( Poster ) > link | Braham Snyder · Yuke Zhu 🔗 |
-
|
Collaborative symmetricity exploitation for offline learning of hardware design solver ( Poster ) > link | HAEYEON KIM · Minsu Kim · joungho kim · Jinkyoo Park 🔗 |
-
|
SPRINT: Scalable Semantic Policy Pre-training via Language Instruction Relabeling
(
Poster
)
>
link
SlidesLive Video |
Jesse Zhang · Karl Pertsch · Jiahui Zhang · Taewook Nam · Sung Ju Hwang · Xiang Ren · Joseph Lim 🔗 |
-
|
Bayesian Q-learning With Imperfect Expert Demonstrations
(
Poster
)
>
link
SlidesLive Video |
Fengdi Che · Xiru Zhu · Doina Precup · David Meger · Gregory Dudek 🔗 |
-
|
Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? ( Poster ) > link | Gunshi Gupta · Tim G. J. Rudner · Rowan McAllister · Adrien Gaidon · Yarin Gal 🔗 |
-
|
Trajectory-based Explainability Framework for Offline RL ( Poster ) > link | Shripad Deshmukh · Arpan Dasgupta · Chirag Agarwal · Nan Jiang · Balaji Krishnamurthy · Georgios Theocharous · Jayakumar Subramanian 🔗 |
-
|
AMORE: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data ( Poster ) > link | Tengyang Xie · Mohak Bhardwaj · Nan Jiang · Ching-An Cheng 🔗 |
-
|
Balanced Off-Policy Evaluation for Personalized Pricing ( Poster ) > link | Adam N. Elmachtoub · Vishal Gupta · YUNFAN ZHAO 🔗 |
-
|
ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learning ( Poster ) > link | Eddy Hudson · Ishan Durugkar · Garrett Warnell · Peter Stone 🔗 |
-
|
Dynamics-Augmented Decision Transformer for Offline Dynamics Generalization ( Poster ) > link | Changyeon Kim · Junsu Kim · Younggyo Seo · Kimin Lee · Honglak Lee · Jinwoo Shin 🔗 |
-
|
Offline Reinforcement Learning on Real Robot with Realistic Data Sources
(
Poster
)
>
link
SlidesLive Video |
Gaoyue Zhou · Liyiming Ke · Siddhartha Srinivasa · Abhinav Gupta · Aravind Rajeswaran · Vikash Kumar 🔗 |
-
|
Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
(
Poster
)
>
link
SlidesLive Video |
Dmitry Akimov · Alexander Nikulin · Vladislav Kurenkov · Denis Tarasov · Sergey Kolesnikov 🔗 |
-
|
Matrix Estimation for Offline Evaluation in Reinforcement Learning with Low-Rank Structure ( Poster ) > link | Xumei Xi · Christina Yu · Yudong Chen 🔗 |
-
|
Train Offline, Test Online: A Real Robot Learning Benchmark
(
Poster
)
>
link
SlidesLive Video |
12 presentersGaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta |
-
|
Hybrid RL: Using both offline and online data can make RL efficient
(
Poster
)
>
link
SlidesLive Video |
Yuda Song · Yifei Zhou · Ayush Sekhari · J. Bagnell · Akshay Krishnamurthy · Wen Sun 🔗 |
-
|
Choreographer: Learning and Adapting Skills in Imagination
(
Poster
)
>
link
SlidesLive Video |
Pietro Mazzaglia · Tim Verbelen · Bart Dhoedt · Alexandre Lacoste · Sai Rajeswar Mudumba 🔗 |
-
|
CORL: Research-oriented Deep Offline Reinforcement Learning Library
(
Poster
)
>
link
SlidesLive Video |
Denis Tarasov · Alexander Nikulin · Dmitry Akimov · Vladislav Kurenkov · Sergey Kolesnikov 🔗 |
-
|
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
(
Poster
)
>
link
SlidesLive Video |
Alexander Nikulin · Vladislav Kurenkov · Denis Tarasov · Dmitry Akimov · Sergey Kolesnikov 🔗 |
-
|
Offline Reinforcement Learning for Customizable Visual Navigation ( Poster ) > link | Dhruv Shah · Arjun Bhorkar · Hrishit Leen · Ilya Kostrikov · Nicholas Rhinehart · Sergey Levine 🔗 |
-
|
Efficient Planning in a Compact Latent Action Space ( Poster ) > link | zhengyao Jiang · Tianjun Zhang · Michael Janner · Yueying (Lisa) Li · Tim Rocktäschel · Edward Grefenstette · Yuandong Tian 🔗 |
-
|
User-Interactive Offline Reinforcement Learning
(
Poster
)
>
link
SlidesLive Video |
Phillip Swazinna · Steffen Udluft · Thomas Runkler 🔗 |
-
|
Does Zero-Shot Reinforcement Learning Exist? ( Poster ) > link | Ahmed Touati · Jérémy Rapin · Yann Ollivier 🔗 |
-
|
State Advantage Weighting for Offline RL ( Poster ) > link | Jiafei Lyu · aicheng Gong · Le Wan · Zongqing Lu · Xiu Li 🔗 |
-
|
Optimal Transport for Offline Imitation Learning
(
Poster
)
>
link
SlidesLive Video |
Yicheng Luo · zhengyao Jiang · Samuel Cohen · Edward Grefenstette · Marc Deisenroth 🔗 |
-
|
Control Graph as Unified IO for Morphology-Task Generalization
(
Poster
)
>
link
SlidesLive Video |
Hiroki Furuta · Yusuke Iwasawa · Yutaka Matsuo · Shixiang (Shane) Gu 🔗 |
-
|
Mutual Information Regularized Offline Reinforcement Learning ( Poster ) > link | Xiao Ma · Bingyi Kang · Zhongwen Xu · Min Lin · Shuicheng Yan 🔗 |
-
|
Uncertainty-Driven Pessimistic Q-Ensemble for Offline-to-Online Reinforcement Learning ( Poster ) > link | Ingook Jang · Seonghyun Kim 🔗 |
-
|
Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling ( Poster ) > link | Ashish Kumar · Ilya Kuzovkin 🔗 |
-
|
Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation ( Poster ) > link | Dan Qiao · Yu-Xiang Wang 🔗 |
-
|
Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Poster
)
>
link
SlidesLive Video |
Jason Yecheng Ma · Shagun Sodhani · Dinesh Jayaraman · Osbert Bastani · Vikash Kumar · Amy Zhang 🔗 |
-
|
Imitation from Observation With Bootstrapped Contrastive Learning ( Poster ) > link | Medric Sonwa · Johanna Hansen · Eugene Belilovsky 🔗 |
-
|
Provable Benefits of Representational Transfer in Reinforcement Learning ( Poster ) > link | Alekh Agarwal · Yuda Song · Kaiwen Wang · Mengdi Wang · Wen Sun · Xuezhou Zhang 🔗 |
-
|
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning ( Poster ) > link | Benjamin Eysenbach · Matthieu Geist · Sergey Levine · Russ Salakhutdinov 🔗 |
-
|
Offline evaluation in RL: soft stability weighting to combine fitted Q-learning and model-based methods ( Poster ) > link | Briton Park · Xian Wu · Bin Yu · Angela Zhou 🔗 |
-
|
Using Confounded Data in Offline RL ( Poster ) > link | Maxime Gasse · Damien GRASSET · Guillaume Gaudron · Pierre-Yves Oudeyer 🔗 |
-
|
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
(
Poster
)
>
link
SlidesLive Video |
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang 🔗 |
-
|
Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based RL
(
Poster
)
>
link
SlidesLive Video |
David Brandfonbrener · Stephen Tu · Avi Singh · Stefan Welker · Chad Boodoo · Nikolai Matni · Jake Varley 🔗 |
-
|
Towards Data-Driven Offline Simulations for Online Reinforcement Learning ( Poster ) > link | Shengpu Tang · Felipe Vieira Frujeri · Dipendra Misra · Alex Lamb · John Langford · Paul Mineiro · Sebastian Kochman 🔗 |
-
|
Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction ( Poster ) > link | Brahma Pavse · Josiah Hanna 🔗 |
-
|
Benchmarking Offline Reinforcement Learning Algorithms for E-Commerce Order Fraud Evaluation ( Poster ) > link | Soysal Degirmenci · Christopher S Jones 🔗 |
-
|
Sparse Q-Learning: Offline Reinforcement Learning with Implicit Value Regularization ( Poster ) > link | Haoran Xu · Li Jiang · Li Jianxiong · Zhuoran Yang · Zhaoran Wang · Xianyuan Zhan 🔗 |