Workshop
5th Robot Learning Workshop: Trustworthy Robotics
Alex Bewley · Roberto Calandra · Anca Dragan · Igor Gilitschenski · Emily Hannigan · Masha Itkina · Hamidreza Kasaei · Jens Kober · Danica Kragic · Nathan Lambert · Julien PEREZ · Fabio Ramos · Ransalu Senanayake · Jonathan Tompson · Vincent Vanhoucke · Markus Wulfmeier
Virtual
Fri 9 Dec, 7 a.m. PST
Machine learning (ML) has been one of the premier drivers of recent advances in robotics research and has made its way into impacting several real-world robotic applications in unstructured and human-centric environments, such as transportation, healthcare, and manufacturing. At the same time, robotics has been a key motivation for numerous research problems in artificial intelligence research, from efficient algorithms to robust generalization of decision models. However, there are still considerable obstacles to fully leveraging state-of-the-art ML in real-world robotics applications. For capable robots equipped with ML models, guarantees on the robustness and additional analysis of the social implications of these models are required for their utilization in real-world robotic domains that interface with humans (e.g. autonomous vehicles, and tele-operated or assistive robots).
To support the development of robots that are safely deployable among humans, the field must consider trustworthiness as a central aspect in the development of real-world robot learning systems. Unlike many other applications of ML, the combined complexity of physical robotic platforms and learning-based perception-action loops presents unique technical challenges. These challenges include concrete technical problems such as very high performance requirements, explainability, predictability, verification, uncertainty quantification, and robust operation in dynamically distributed, open-set domains. Since robots are developed for use in human environments, in addition to these technical challenges, we must also consider the social aspects of robotics such as privacy, transparency, fairness, and algorithmic bias. Both technical and social challenges also present opportunities for robotics and ML researchers alike. Contributing to advances in the aforementioned sub-fields promises to have an important impact on real-world robot deployment in human environments, building towards robots that use human feedback, indicate when their model is uncertain, and are safe to operate autonomously in safety-critical settings such as healthcare and transportation.
This year’s robot learning workshop aims at discussing unique research challenges from the lens of trustworthy robotics. We adopt a broad definition of trustworthiness that highlights different application domains and the responsibility of the robotics and ML research communities to develop “robots for social good.” Bringing together experts with diverse backgrounds from the ML and robotics communities, the workshop will offer new perspectives on trust in the context of ML-driven robot systems.
Scope of contributions:
Specific areas of interest include but are not limited to:
* epistemic uncertainty estimation in robotics;
* explainable robot learning;
* domain adaptation and distribution shift in robot learning;
* multi-modal trustworthy sensing and sensor fusion;
* safe deployment for applications such as agriculture, space, science, and healthcare;
* privacy aware robotic perception;
* information system security in robot learning;
* learning from offline data and safe on-line learning;
* simulation-to-reality transfer for safe deployment;
* robustness and safety evaluation;
* certifiability and performance guarantees;
* robotics for social good;
* safe robot learning with humans in the loop;
* algorithmic bias in robot learning;
* ethical robotics.
Schedule
Fri 7:00 a.m. - 7:15 a.m.
|
Opening Remarks
(
Workshop Introduction
)
>
SlidesLive Video |
🔗 |
Fri 7:15 a.m. - 7:30 a.m.
|
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
(
Contributed Talk 1
)
>
|
Ivan Kapelyukh · Vitalis Vosylius · Edward Johns 🔗 |
Fri 7:30 a.m. - 8:15 a.m.
|
Panel: Uncertainty-Aware Machine Learning for Robotics (Q&A 1)
(
Discussion Panel
)
>
SlidesLive Video |
Georgia Chalvatzaki · Stefanie Tellex · Animesh Garg 🔗 |
Fri 8:15 a.m. - 8:30 a.m.
|
Coffee Break 1
|
🔗 |
Fri 8:30 a.m. - 9:15 a.m.
|
Panel: Scaling & Models (Q&A 2)
(
Discussion Panel
)
>
SlidesLive Video |
Andy Zeng · Haoran Tang · Karol Hausman · Jackie Kay · Gabriel Barth-Maron 🔗 |
Fri 9:15 a.m. - 10:15 a.m.
|
Poster Session 1
(
Poster Session
)
>
|
🔗 |
Fri 10:15 a.m. - 11:00 a.m.
|
Panel: Safety and Verification for Decision-Making Systems (Q&A 3)
(
Discussion Panel
)
>
SlidesLive Video |
Luca Carlone · Sarah Dean · Matthew Johnson-Roberson 🔗 |
Fri 11:00 a.m. - 3:00 p.m.
|
Long Break
|
🔗 |
Fri 3:00 p.m. - 4:00 p.m.
|
Debate: Robotics for Good
(
Discussion Panel
)
>
SlidesLive Video |
Karol Hausman · Katherine Driggs-Campbell · Luca Carlone · Sarah Dean · Matthew Johnson-Roberson · Animesh Garg 🔗 |
Fri 4:00 p.m. - 4:15 p.m.
|
Robust Forecasting for Robotic Control: A Game-Theoretic Approach
(
Contributed Talk 2
)
>
|
Shubhankar Agarwal · David Fridovich-Keil · Sandeep Chinchali 🔗 |
Fri 4:15 p.m. - 4:30 p.m.
|
Certifiably-correct Control Policies for Safe Learning and Adaptation in Assistive Robotics
(
Contributed Talk 3
)
>
|
Keyvan Majd · GEOFFEY CLARK · Tanmay Khandait · Heni Ben Amor 🔗 |
Fri 4:45 p.m. - 5:00 p.m.
|
Coffee Break 2
|
🔗 |
Fri 5:00 p.m. - 6:00 p.m.
|
Poster Session 2
(
Poster Session
)
>
|
🔗 |
Fri 6:00 p.m. - 6:45 p.m.
|
Panel: Explainability/Predictability Robotics (Q&A 4)
(
Discussion Panel
)
>
SlidesLive Video |
Katherine Driggs-Campbell · Been Kim · Leila Takayama 🔗 |
Fri 6:45 p.m. - 7:00 p.m.
|
Closing Remarks
(
Workshop Presentation
)
>
SlidesLive Video |
🔗 |
-
|
Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based RL
(
Poster
)
>
SlidesLive Video |
David Brandfonbrener · Stephen Tu · Avi Singh · Stefan Welker · Chad Boodoo · Nikolai Matni · Jacob Varley 🔗 |
-
|
Conformal Semantic Keypoint Detection with Statistical Guarantees
(
Poster
)
>
SlidesLive Video |
Heng Yang · Marco Pavone 🔗 |
-
|
A Contextual Bandit Approach for Learning to Plan in Environments with Probabilistic Goal Configurations
(
Poster
)
>
SlidesLive Video |
Sohan Rudra · Saksham Goel · Anirban Santara · Claudio Gentile · Laurent Perron · Fei Xia · Vikas Sindhwani · Carolina Parada · Gaurav Aggarwal 🔗 |
-
|
Imitating careful experts to avoid catastrophic events
(
Poster
)
>
SlidesLive Video |
Jack Hanslope · Laurence Aitchison 🔗 |
-
|
Infrastructure-based End-to-End Learning and Prevention of Driver Failure
(
Poster
)
>
SlidesLive Video |
Noam Buckman · Shiva Sreeram · Mathias Lechner · Yutong Ban · Ramin Hasani · Sertac Karaman · Daniela Rus 🔗 |
-
|
Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty
(
Poster
)
>
SlidesLive Video |
Thom Badings · Licio Romao · Alessandro Abate · Nils Jansen 🔗 |
-
|
A Benchmark for Out of Distribution Detection in Point Cloud 3D Semantic Segmentation
(
Poster
)
>
SlidesLive Video |
Lokesh Veeramacheneni · Matias Valdenegro-Toro 🔗 |
-
|
VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Poster
)
>
SlidesLive Video |
Jason Yecheng Ma · Shagun Sodhani · Dinesh Jayaraman · Osbert Bastani · Vikash Kumar · Amy Zhang 🔗 |
-
|
Learning Certifiably Robust Controllers Using Fragile Perception
(
Poster
)
>
|
Dawei Sun · Negin Musavi · Geir Dullerud · Sanjay Shakkottai · Sayan Mitra 🔗 |
-
|
PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning
(
Poster
)
>
SlidesLive Video |
Jelle Luijkx · Zlatan Ajanovic · Laura Ferranti · Jens Kober 🔗 |
-
|
Robust Forecasting for Robotic Control: A Game-Theoretic Approach
(
Poster
)
>
SlidesLive Video |
Shubhankar Agarwal · David Fridovich-Keil · Sandeep Chinchali 🔗 |
-
|
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
(
Poster
)
>
SlidesLive Video |
Ivan Kapelyukh · Vitalis Vosylius · Edward Johns 🔗 |
-
|
MAEA: Multimodal Attribution Framework for Embodied AI
(
Poster
)
>
SlidesLive Video |
Vidhi Jain · Jayant Sravan Tamarapalli · Sahiti Yerramilli · Yonatan Bisk 🔗 |
-
|
Safety-Guaranteed Skill Discovery for Robot Manipulation Tasks
(
Poster
)
>
SlidesLive Video |
Sunin Kim · Jaewoon Kwon · Taeyoon Lee · Younghyo Park · Julien PEREZ 🔗 |
-
|
Insights towards Sim2Real Contact-Rich Manipulation
(
Poster
)
>
SlidesLive Video |
Michael Noseworthy · Iretiayo Akinola · Yashraj Narang · Fabio Ramos · Lucas Manuelli · Ankur Handa · Dieter Fox 🔗 |
-
|
Train Offline, Test Online: A Real Robot Learning Benchmark
(
Poster
)
>
SlidesLive Video |
12 presentersGaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta |
-
|
Learning a Meta-Controller for Dynamic Grasping
(
Poster
)
>
SlidesLive Video |
Yinsen Jia · Jingxi Xu · Dinesh Jayaraman · Shuran Song 🔗 |
-
|
Real World Offline Reinforcement Learning with Realistic Data Source
(
Poster
)
>
SlidesLive Video |
Gaoyue Zhou · Liyiming Ke · Siddhartha Srinivasa · Abhinav Gupta · Aravind Rajeswaran · Vikash Kumar 🔗 |
-
|
Interactive Language: Talking to Robots in Real Time
(
Poster
)
>
SlidesLive Video |
Corey Lynch · Pete Florence · Jonathan Tompson · Ayzaan Wahid · Tianli Ding · James Betker · Robert Baruch · Travis Armstrong 🔗 |
-
|
Robotic Skill Acquistion via Instruction Augmentation with Vision-Language Models
(
Poster
)
>
SlidesLive Video |
Ted Xiao · Harris Chan · Pierre Sermanet · Ayzaan Wahid · Anthony Brohan · Karol Hausman · Sergey Levine · Jonathan Tompson 🔗 |
-
|
Certifiably-correct Control Policies for Safe Learning and Adaptation in Assistive Robotics
(
Poster
)
>
|
Keyvan Majd · GEOFFEY CLARK · Tanmay Khandait · · Sriram Sankaranarayanan · Georgios Fainekos · Heni Ben Amor 🔗 |
-
|
Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks
(
Poster
)
>
SlidesLive Video |
Sadhana Lolla · Iaroslav Elistratov · Alejandro Perez · Elaheh Ahmadi · Daniela Rus · Alexander Amini 🔗 |
-
|
Language as Robot Middleware - Andy Zeng & Jacky Liang
(
Discussion Panel: Scaling + Models
)
>
link
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Angela Schoellig
(
Discussion Panel: Uncertainty-Aware ML for Robotics
)
>
|
Hamidreza Kasaei 🔗 |
-
|
Uncertainty Aware Machine Learning for Robotics - Stefanie Tellex
(
Discussion Panel: Uncertainty-Aware ML for Robotics
)
>
|
Hamidreza Kasaei 🔗 |
-
|
Real Robots Learn with Structure - Georgia Chalvatzaki ( Discussion Panel: Uncertainty-Aware ML for Robotics ) > link | Hamidreza Kasaei 🔗 |
-
|
My Hopes and Dreams of Communicating with Machines and Where to Begin - Beem Kim ( Discussion Panel: Explainability/Predictability in Robotics ) > link | Hamidreza Kasaei 🔗 |
-
|
Why Robots Need Social Skills - Leila Takayama
(
Discussion Panel: Explainability/Predictability in Robotics
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Representing Interactions for Robot Navigation - Katherine Driggs-Campbell & Zhe Huang
(
Discussion Panel: Explainability/Predictability in Robotics
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Towards Certifiably Safe Nonlinear Control with Sensor and Dynamics Uncertainties - Sarah Dean & Andrew Taylor
(
Discussion Panel: Safety & Verification for Decision-Making
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
A Vision for Certifiable Perception: from Outlier-Robust Estimation to Self-Supervised Learning - Luca Carlone & Rajat Talak
(
Discussion Panel: Safety & Verification for Decision-Making
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Self-driving Cars - Matthew Johnson-Roberson
(
Discussion Panel: Safety & Verification for Decision-Making
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Trustworthy AI Robotics for Real-world Logistics - Anusha Nagabandi
(
Discussion Panel: Scaling + Models
)
>
link
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
A Generalist Agent (GATO) - Scott Reed & Gabriel Barth-Maron & Jackie Kay
(
Discussion Panel: Scaling + Models
)
>
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Language as a Connective Tissue for Robotics - Karol Hausman & Brian Ichter
(
Discussion Panel: Safety & Verification for Decision-Making
)
>
link
SlidesLive Video |
Hamidreza Kasaei 🔗 |
-
|
Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based RL
(
Oral
)
>
|
David Brandfonbrener · Stephen Tu · Avi Singh · Stefan Welker · Chad Boodoo · Nikolai Matni · Jacob Varley 🔗 |
-
|
Conformal Semantic Keypoint Detection with Statistical Guarantees
(
Oral
)
>
|
Heng Yang · Marco Pavone 🔗 |
-
|
A Contextual Bandit Approach for Learning to Plan in Environments with Probabilistic Goal Configurations
(
Oral
)
>
|
Sohan Rudra · Saksham Goel · Anirban Santara · Claudio Gentile · Laurent Perron · Fei Xia · Vikas Sindhwani · Carolina Parada · Gaurav Aggarwal 🔗 |
-
|
Imitating careful experts to avoid catastrophic events
(
Oral
)
>
|
Jack Hanslope · Laurence Aitchison 🔗 |
-
|
Infrastructure-based End-to-End Learning and Prevention of Driver Failure
(
Oral
)
>
|
Noam Buckman · Shiva Sreeram · Mathias Lechner · Yutong Ban · Ramin Hasani · Sertac Karaman · Daniela Rus 🔗 |
-
|
Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty
(
Oral
)
>
|
Thom Badings · Licio Romao · Alessandro Abate · Nils Jansen 🔗 |
-
|
A Benchmark for Out of Distribution Detection in Point Cloud 3D Semantic Segmentation
(
Oral
)
>
|
Lokesh Veeramacheneni · Matias Valdenegro-Toro 🔗 |
-
|
VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Oral
)
>
|
Jason Yecheng Ma · Shagun Sodhani · Dinesh Jayaraman · Osbert Bastani · Vikash Kumar · Amy Zhang 🔗 |
-
|
Learning Certifiably Robust Controllers Using Fragile Perception
(
Oral
)
>
|
Dawei Sun · Negin Musavi · Geir Dullerud · Sanjay Shakkottai · Sayan Mitra 🔗 |
-
|
PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning
(
Oral
)
>
|
Jelle Luijkx · Zlatan Ajanovic · Laura Ferranti · Jens Kober 🔗 |
-
|
MAEA: Multimodal Attribution Framework for Embodied AI
(
Oral
)
>
|
Vidhi Jain · Jayant Sravan Tamarapalli · Sahiti Yerramilli · Yonatan Bisk 🔗 |
-
|
Safety-Guaranteed Skill Discovery for Robot Manipulation Tasks
(
Oral
)
>
|
Sunin Kim · Jaewoon Kwon · Taeyoon Lee · Younghyo Park · Julien PEREZ 🔗 |
-
|
Insights towards Sim2Real Contact-Rich Manipulation
(
Oral
)
>
|
Michael Noseworthy · Iretiayo Akinola · Yashraj Narang · Fabio Ramos · Lucas Manuelli · Ankur Handa · Dieter Fox 🔗 |
-
|
Train Offline, Test Online: A Real Robot Learning Benchmark
(
Oral
)
>
|
12 presentersGaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta |
-
|
Learning a Meta-Controller for Dynamic Grasping
(
Oral
)
>
|
Yinsen Jia · Jingxi Xu · Dinesh Jayaraman · Shuran Song 🔗 |
-
|
Real World Offline Reinforcement Learning with Realistic Data Source
(
Oral
)
>
|
Gaoyue Zhou · Liyiming Ke · Siddhartha Srinivasa · Abhinav Gupta · Aravind Rajeswaran · Vikash Kumar 🔗 |
-
|
Interactive Language: Talking to Robots in Real Time
(
Oral
)
>
|
Corey Lynch · Pete Florence · Jonathan Tompson · Ayzaan Wahid · Tianli Ding · James Betker · Robert Baruch · Travis Armstrong 🔗 |
-
|
Robotic Skill Acquistion via Instruction Augmentation with Vision-Language Models
(
Oral
)
>
|
Ted Xiao · Harris Chan · Pierre Sermanet · Ayzaan Wahid · Anthony Brohan · Karol Hausman · Sergey Levine · Jonathan Tompson 🔗 |
-
|
Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks
(
Oral
)
>
|
Sadhana Lolla · Iaroslav Elistratov · Alejandro Perez · Elaheh Ahmadi · Daniela Rus · Alexander Amini 🔗 |
-
|
Robust Forecasting for Robotic Control: A Game-Theoretic Approach
(
Oral
)
>
|
Shubhankar Agarwal · David Fridovich-Keil · Sandeep Chinchali 🔗 |
-
|
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
(
Oral
)
>
|
Ivan Kapelyukh · Vitalis Vosylius · Edward Johns 🔗 |
-
|
Certifiably-correct Control Policies for Safe Learning and Adaptation in Assistive Robotics
(
Oral
)
>
|
Keyvan Majd · GEOFFEY CLARK · Tanmay Khandait · · Sriram Sankaranarayanan · Georgios Fainekos · Heni Ben Amor 🔗 |