Workshop
Differentiable computer vision, graphics, and physics in machine learning
Krishna Murthy Jatavallabhula · Kelsey Allen · Victoria Dean · Johanna Hansen · Shuran Song · Florian Shkurti · Liam Paull · Derek Nowrouzezahrai · Josh Tenenbaum
Fri 11 Dec, 6:45 a.m. PST
“Differentiable programs” are parameterized programs that allow themselves to be rewritten by gradient-based optimization. They are ubiquitous in modern-day machine learning. Recently, explicitly encoding our knowledge of the rules of the world in the form of differentiable programs has become more popular. In particular, differentiable realizations of well-studied processes such as physics, rendering, projective geometry, optimization to name a few, have enabled the design of several novel learning techniques. For example, many approaches have been proposed for unsupervised learning of depth estimation from unlabeled videos. Differentiable 3D reconstruction pipelines have demonstrated the potential for task-driven representation learning. A number of differentiable rendering approaches have been shown to enable single-view 3D reconstruction and other inverse graphics tasks (without requiring any form of 3D supervision). Differentiable physics simulators are being built to perform physical parameter estimation from video or for model-predictive control. While these advances have largely occurred in isolation, recent efforts have attempted to bridge the gap between the aforementioned areas. Narrowing the gaps between these otherwise isolated disciplines holds tremendous potential to yield new research directions and solve long-standing problems, particularly in understanding and reasoning about the 3D world.
Hence, we propose the “first workshop on differentiable computer vision, graphics, and physics in machine learning” with the aim of:
1. Narrowing the gap and fostering synergies between the computer vision, graphics, physics, and machine learning communities
2. Debating the promise and perils of differentiable methods, and identifying challenges that need to be overcome
3. Raising awareness about these techniques to the larger ML community
4. Discussing the broader impact of such techniques, and any ethical implications thereof.
Schedule
Fri 6:45 a.m. - 7:00 a.m.
|
Opening remarks
(
Introductory remarks
)
>
|
Krishna Murthy Jatavallabhula · Kelsey Allen · Johanna Hansen · Victoria Dean 🔗 |
Fri 7:00 a.m. - 7:30 a.m.
|
Sanja Fidler
(
Invited talk
)
>
|
Sanja Fidler 🔗 |
Fri 7:30 a.m. - 8:00 a.m.
|
Andrea Tagliasacchi
(
Invited talk
)
>
|
Andrea Tagliasacchi 🔗 |
Fri 8:02 a.m. - 8:31 a.m.
|
Peter Battaglia
(
Invited talk
)
>
SlidesLive Video |
Peter Battaglia 🔗 |
Fri 8:32 a.m. - 8:37 a.m.
|
Peter Battaglia - Q&A
(
Q&A
)
>
|
🔗 |
Fri 8:38 a.m. - 8:53 a.m.
|
Camillo Jose Taylor
(
Invited talk
)
>
SlidesLive Video |
Camillo Taylor 🔗 |
Fri 8:54 a.m. - 8:59 a.m.
|
Camillo Jose Taylor - Q&A
(
Q&A
)
>
|
🔗 |
Fri 9:00 a.m. - 9:13 a.m.
|
Oral 01: phiflow - A differentiable PDE solving framework for deep learning via physical simulations
(
Contributed Talk
)
>
SlidesLive Video |
Nils Thuerey 🔗 |
Fri 9:13 a.m. - 9:23 a.m.
|
Oral 02: Differentiable HDR image synthesis using multi-exposure images
(
Contributed Talk
)
>
SlidesLive Video |
Jung Hee Kim 🔗 |
Fri 9:23 a.m. - 9:35 a.m.
|
Oral 03: DELUCA - Differentiable control library - environments, methods, and benchmarking
(
Contributed Talk
)
>
SlidesLive Video |
Paula Gradu 🔗 |
Fri 9:35 a.m. - 9:44 a.m.
|
Oral 04: Blendshape-augmented facial action units detection
(
Contributed Talk
)
>
SlidesLive Video |
Zijun Cui 🔗 |
Fri 9:44 a.m. - 9:57 a.m.
|
Oral 05: Inverse articulated-body dynamics from video via variational sequential Monte-Carlo
(
Contributed Talk
)
>
SlidesLive Video |
Dan Biderman 🔗 |
Fri 9:58 a.m. - 10:08 a.m.
|
Contributed Talk - Q&A
(
Q&A
)
>
|
🔗 |
Fri 10:10 a.m. - 10:35 a.m.
|
Bethany Lusch
(
Invited talk
)
>
SlidesLive Video |
Bethany Lusch 🔗 |
Fri 10:36 a.m. - 10:41 a.m.
|
Bethany Lusch - Q&A
(
Q&A
)
>
|
🔗 |
Fri 10:42 a.m. - 11:13 a.m.
|
Yuanming Hu
(
Invited talk
)
>
SlidesLive Video |
Yuanming Hu 🔗 |
Fri 11:14 a.m. - 11:19 a.m.
|
Yuanming Hu - Q&A
(
Q&A
)
>
|
🔗 |
Fri 11:20 a.m. - 11:39 a.m.
|
Georgia Gkioxari
(
Invited talk
)
>
SlidesLive Video |
Georgia Gkioxari 🔗 |
Fri 11:40 a.m. - 11:45 a.m.
|
Georgia Gkioxari - Q&A
(
Q&A
)
>
|
🔗 |
Fri 11:46 a.m. - 12:16 p.m.
|
Ming Lin
(
Invited talk
)
>
|
Ming Lin 🔗 |
Fri 12:16 p.m. - 1:15 p.m.
|
Panel Discussion
(
Panel Discussion
)
>
|
🔗 |
Fri 1:15 p.m. - 2:30 p.m.
|
Poster session (gather.town) ( Poster Session ) > link | 🔗 |
-
|
Poster 01: Using differentiable physics for self-supervised assimilation of chaotic dynamical systems
(
Contributed poster
)
>
SlidesLive Video |
John McCabe 🔗 |
-
|
Poster 02: Learned equivariant rendering without transformation supervision
(
Contributed poster
)
>
SlidesLive Video |
Cinjon Resnick 🔗 |
-
|
Poster 03: Differentiable data augmentation with Kornia
(
Contributed poster
)
>
SlidesLive Video |
Jian Shi 🔗 |
-
|
Poster 04: Semantic adversarial robustness with differentiable ray-tracing
(
Contributed poster
)
>
SlidesLive Video |
Rahul Venkatesh 🔗 |
-
|
Poster 05: Inverse graphics GAN
(
Contributed poster
)
>
SlidesLive Video |
Sebastian Lunz 🔗 |
-
|
Poster 06: Instance-wise depth and motion learning from monocular videos
(
Contributed poster
)
>
SlidesLive Video |
Seokju Lee 🔗 |
-
|
Poster 07: System level differentiable simulation of radio access networks
(
Contributed poster
)
>
SlidesLive Video |
Dmitriy Rivkin 🔗 |
-
|
Poster 08: Solving physics puzzles by reasoning about paths
(
Contributed poster
)
>
SlidesLive Video |
Augustin Harter 🔗 |
-
|
Poster 09: Sparse-input neural network augmentations for differentiable simulators
(
Contributed poster
)
>
SlidesLive Video |
Eric Heiden · David Millard 🔗 |
-
|
Poster 10: Tractable loss function and color image generation of multinary restricted Boltzmann machine
(
Contributed poster
)
>
SlidesLive Video |
Juno Hwang 🔗 |
-
|
Poster 11: Differentiable path tracing by regularizing discontinuities
(
Contributed poster
)
>
SlidesLive Video |
Peter Quinn 🔗 |
-
|
Poster 12: Spring-Rod system identification via differentiable physics engine
(
Contributed poster
)
>
SlidesLive Video |
Kun Wang 🔗 |
-
|
Poster 13: End-to-end differentiable 6DoF object pose estimation with local and global constraints
(
Contributed poster
)
>
SlidesLive Video |
Anshul Gupta · Joydeep Medhi · Aratrik Chattopadhyay · Vikram Gupta 🔗 |
-
|
Poster 14: MSR-Net: Multi-scale relighting network for one-to-one relighting
(
Contributed poster
)
>
SlidesLive Video |
Nisarg Shah 🔗 |
-
|
Poster 15: Towards end-to-end training of proposal-based 3D human pose estimation
(
Contributed poster
)
>
SlidesLive Video |
Daniel Ajisafe 🔗 |