Workshop
NeurIPS 2023 Workshop on Diffusion Models
Bahjat Kawar · Valentin De Bortoli · Charlotte Bunne · James Thornton · Jiaming Song · Jong Chul Ye · Chenlin Meng
Hall B1 (level 1)
Fri 15 Dec, 6:50 a.m. PST
Over the past three years, diffusion models have established themselves as a new generative modeling paradigm. Their empirical successes have broadened the applications of generative modeling to image, video, audio, 3D synthesis, science applications, and more. As diffusion models become more and more popular and are applied to extremely diverse problems, it also becomes harder to follow the key contributions in the field. This workshop aims to keep track of recent advances and identify guidelines for future research. By bringing together practice, methodology, and theory actors we aim to identify unexplored areas, foster collaboration, and push the frontier of diffusion model research.
Link to website: https://diffusionworkshop.github.io/
Ask questions to our panelists here: https://docs.google.com/forms/d/e/1FAIpQLSeTRsWFvKlsFg31K8Vq6hHGOydmvd7YNMuOLOCcKgqSqO8mXw/viewform
Schedule
Fri 6:50 a.m. - 7:00 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
|
🔗 |
Fri 7:00 a.m. - 7:30 a.m.
|
Tali Dekel: Pretrained diffusion is all we need: a journey beyond training distribution
(
Keynote
)
>
SlidesLive Video |
🔗 |
Fri 7:30 a.m. - 8:00 a.m.
|
Brian Trippe & Jason Yim: De novo design of protein structure and function with RFdiffusion
(
Keynote
)
>
SlidesLive Video |
🔗 |
Fri 8:00 a.m. - 8:15 a.m.
|
Coffee Break
|
🔗 |
Fri 8:15 a.m. - 9:15 a.m.
|
Poster Session 1
(
Poster session
)
>
|
🔗 |
Fri 9:15 a.m. - 10:00 a.m.
|
Panel Discussion: Arash Vahdat, Ruiqi Gao, Tim Salimans, Robin Rombach
(
Panel Discussion
)
>
SlidesLive Video |
🔗 |
Fri 10:00 a.m. - 11:30 a.m.
|
Lunch
|
🔗 |
Fri 11:30 a.m. - 12:00 p.m.
|
Sayak Paul: Controlling Text-to-Image Diffusion Models
(
Keynote
)
>
SlidesLive Video |
🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Yang Song: Consistency Models
(
Keynote
)
>
SlidesLive Video |
🔗 |
Fri 12:30 p.m. - 12:40 p.m.
|
LC-SD: Realistic Endoscopic Image Generation with Stable Diffusion and ControlNet
(
Oral
)
>
link
SlidesLive Video |
Joanna Kaleta · Diego Dall'Alba · Szymon Plotka · Przemyslaw Korzeniowski 🔗 |
Fri 12:40 p.m. - 12:50 p.m.
|
Manifold Diffusion Fields
(
Oral
)
>
link
SlidesLive Video |
Ahmed Elhag · Yuyang Wang · Joshua Susskind · Miguel Angel Bautista 🔗 |
Fri 12:50 p.m. - 1:00 p.m.
|
The Emergence of Reproducibility and Consistency in Diffusion Models
(
Oral
)
>
link
SlidesLive Video |
Huijie Zhang · Jinfan Zhou · Yifu Lu · Minzhe Guo · Liyue Shen · Qing Qu 🔗 |
Fri 1:00 p.m. - 1:15 p.m.
|
Coffee Break
|
🔗 |
Fri 1:15 p.m. - 2:15 p.m.
|
Invited Lightning Talks
(
Lightning Talks
)
>
SlidesLive Video |
🔗 |
Fri 2:15 p.m. - 3:15 p.m.
|
Poster Session 2
(
Poster Session
)
>
|
🔗 |
Fri 3:15 p.m. - 3:30 p.m.
|
Closing Remarks + Award Reveal
(
Closing Remarks
)
>
SlidesLive Video |
🔗 |
-
|
TS-DiffuGen: An equivariant diffusion model for reaction transition state conformation generation ( Poster ) > link | Sacha Raffaud · Jeff Guo · Philippe Schwaller 🔗 |
-
|
EDGE++: Improved Training and Sampling of EDGE ( Poster ) > link | Xiaohui Chen · Mingyang Wu · Liping Liu 🔗 |
-
|
The Hidden Linear Structure in Score-Based Models and its Application ( Poster ) > link | Binxu Wang · John Vastola 🔗 |
-
|
Controllable Music Production with Diffusion Models and Guidance Gradients ( Poster ) > link | Mark Levy · Bruno Di Giorgi · Floris Weers · Angelos Katharopoulos · Tom R Nickson 🔗 |
-
|
Effective Quantization for Diffusion Models on CPUs ( Poster ) > link | Hanwen Chang · Haihao Shen · Yiyang Cai · Xinyu Ye · Zhenzhong Xu · Wenhua Cheng · Weiwei Zhang · Yintong Lu · Heng Guo 🔗 |
-
|
Staged Diffusion Models with Analytically Designed Hyperparameters ( Poster ) > link | Kanad Pardeshi · Shrey Singla · Sunita Sarawagi 🔗 |
-
|
VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing ( Poster ) > link | Paul Couairon · Clément Rambour · Jean-Emmanuel HAUGEARD · Nicolas THOME 🔗 |
-
|
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion ( Poster ) > link | Dongjun Kim · Chieh-Hsin Lai · Wei-Hsiang Liao · Naoki Murata · Yuhta Takida · Toshimitsu Uesaka · Yutong He · Yuki Mitsufuji · Stefano Ermon 🔗 |
-
|
Learning from Invalid Data: On Constraint Satisfaction in Generative Models ( Poster ) > link | Giorgio Giannone · Lyle Regenwetter · Akash Srivastava · Dan Gutfreund · Faez Ahmed 🔗 |
-
|
Diffusion-C: Unveiling the Generative Challenges of Diffusion Models through Corrupted Data ( Poster ) > link | Keywoong Bae · Suan Lee · Wookey Lee 🔗 |
-
|
Beyond Generation: Exploring Generalization of Diffusion Models in Few-shot Segmentation ( Poster ) > link | Jie Liu · TAO HU · Jan-jakob Sonke · Efstratios Gavves 🔗 |
-
|
Improving Discrete Diffusion Models via Structured Preferential Generation ( Poster ) > link | Severi Rissanen · Markus Heinonen · Arno Solin 🔗 |
-
|
Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization ( Poster ) > link | Yinbin Han · Meisam Razaviyayn · Renyuan Xu 🔗 |
-
|
Decoding the Seeing Brain: Reconstructing Images and Text from fMRI Recordings with multimodal diffusion models ( Poster ) > link | Matteo Ferrante · Tommaso Boccato · Furkan Ozcelik · Rufin VanRullen · Nicola Toschi 🔗 |
-
|
Fast Sampling via De-randomization for Discrete Diffusion Models ( Poster ) > link | Zixiang Chen · Angela Yuan · Yongqian Li · Yiwen Kou · Junkai Zhang · Quanquan Gu 🔗 |
-
|
Diffusion models for probabilistic programming ( Poster ) > link | Simon Dirmeier · Fernando Perez-Cruz 🔗 |
-
|
A Denoising Diffusion Model for Synthetic Fluid Field Prediction ( Poster ) > link | Gefan Yang · Stefan Sommer 🔗 |
-
|
AlphaFold Meets Flow Matching for Generating Protein Ensembles ( Poster ) > link | Bowen Jing · Bonnie Berger · Tommi Jaakkola 🔗 |
-
|
SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions ( Poster ) > link | Yuseung Lee · Kunho Kim · Hyunjin Kim · Minhyuk Sung 🔗 |
-
|
Retrofitting 2D Latent Diffusion for Controllable Face Image Generation ( Poster ) > link | Weihao Xia · Cengiz Oztireli · Jing-Hao Xue 🔗 |
-
|
IterInv: Iterative Inversion for Pixel-Level T2I Models ( Poster ) > link | Chuanming Tang · kai wang · Joost van de Weijer 🔗 |
-
|
Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE ( Poster ) > link | Marius Arvinte · Cory Cornelius · Jason Martin · Nageen Himayat 🔗 |
-
|
Towards Fast Stochastic Sampling in Diffusion Generative Models ( Poster ) > link | Kushagra Pandey · Maja Rudolph · Stephan Mandt 🔗 |
-
|
Motion Flow Matching for Efficient Human Motion Synthesis and Editing ( Poster ) > link | TAO HU · Wenzhe Yin · Pingchuan Ma · Yunlu Chen · Basura Fernando · Yuki M Asano · Efstratios Gavves · Pascal Mettes · Björn Ommer · Cees Snoek 🔗 |
-
|
In search of dispersed memories: Generative diffusion models are associative memory networks ( Poster ) > link | Luca Ambrogioni 🔗 |
-
|
Improved Convergence of Score-Based Diffusion Models via Prediction-Correction ( Poster ) > link | Francesco Pedrotti · Jan Maas · Marco Mondelli 🔗 |
-
|
You Only Submit One Image to Find the Most Suitable Generative Model ( Poster ) > link | Zhi Zhou · Lan-Zhe Guo · Pengxiao Song · Yu-Feng Li 🔗 |
-
|
Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planning ( Poster ) > link | Junwoo Chang · Hyunwoo Ryu · Jiwoo Kim · Soochul Yoo · Joohwan Seo · Nikhil Potu Surya Prakash · Jongeun Choi · Roberto Horowitz 🔗 |
-
|
DiffEnc: Variational Diffusion with a Learned Encoder ( Poster ) > link | Beatrix M. G. Nielsen · Anders Christensen · Andrea Dittadi · Ole Winther 🔗 |
-
|
Beyond U: Making Diffusion Models Faster & Lighter ( Poster ) > link | Sergio Calvo Ordoñez · Jiahao Huang · Lipei Zhang · Guang Yang · Carola-Bibiane Schönlieb · ANGELICA I AVILES-RIVERO 🔗 |
-
|
Understanding Denoising Diffusion Probabilistic Models and their Noise Schedules via the Ornstein--Uhlenbeck Process ( Poster ) > link | Javier E. Santos · Yen Ting Lin 🔗 |
-
|
Diffusion based Zero-shot Medical Image-to-Image Translation for Cross Modality Segmentation ( Poster ) > link | zihao wang · YINGYU YANG · Yuzhou Chen · Tingting Yuan · Maxime Sermesant · Herve Delingette · Ona Wu 🔗 |
-
|
Harmonic Prior Self-conditioned Flow Matching for Multi-Ligand Docking and Binding Site Design ( Poster ) > link | Hannes Stärk · Bowen Jing · Regina Barzilay · Tommi Jaakkola 🔗 |
-
|
Sharp analysis of learning a flow-based generative model from limited sample complexity ( Poster ) > link | Hugo Cui · Eric Vanden-Eijnden · Florent Krzakala · Lenka Zdeborová 🔗 |
-
|
Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective ( Poster ) > link | Davide Ghio · Yatin Dandi · Florent Krzakala · Lenka Zdeborová 🔗 |
-
|
Enhancing Diffusion-based Point Cloud Generation with Smoothness Constraint ( Poster ) > link | Yukun Li · Liping Liu 🔗 |
-
|
Circumventing Concept Erasure Methods For Text-to-Image Generative Models ( Poster ) > link | Minh Pham · Kelly Marshall · Niv Cohen · Govind Mittal · Chinmay Hegde 🔗 |
-
|
Latent Diffusion for Document Generation with Sequential Decoding ( Poster ) > link | Zihuiwen Ye · Elle Michelle Yang · Phil Blunsom 🔗 |
-
|
The Emergence of Reproducibility and Consistency in Diffusion Models ( Poster ) > link | Huijie Zhang · Jinfan Zhou · Yifu Lu · Minzhe Guo · Liyue Shen · Qing Qu 🔗 |
-
|
Exploring Attribute Variations in Style-based GANs using Diffusion Models ( Poster ) > link | Rishubh Parihar · Prasanna Balaji · Raghav Magazine · Sarthak Vora · Tejan Karmali · Varun Jampani · Venkatesh Babu R 🔗 |
-
|
LDM3D-VR: Latent Diffusion Model for 3D VR ( Poster ) > link | Gabriela Ben-Melech Stan · Diana Wofk · Estelle Aflalo · Shao-Yen Tseng · zhipeng cai · Michael Paulitsch · VASUDEV LAL 🔗 |
-
|
DiffDock-Pocket: Diffusion for Pocket-Level Docking with Sidechain Flexibility ( Poster ) > link | Michael Plainer · Marcella Toth · Simon Dobers · Hannes Stärk · Gabriele Corso · Céline Marquet · Regina Barzilay 🔗 |
-
|
Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples ( Poster ) > link | Phillip Howard · Avinash Madasu · Tiep Le · Gustavo Lujan-Moreno · VASUDEV LAL 🔗 |
-
|
WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models ( Poster ) > link | Changhoon Kim · Kyle Min · Maitreya Patel · Sheng Cheng · 'YZ' Yezhou Yang 🔗 |
-
|
$f$-GANs Settle Scores! ( Poster ) > link | Siddarth Asokan · Nishanth Shetty · Aadithya Srikanth · Chandra Seelamantula 🔗 |
-
|
Latent Painter ( Poster ) > link | Shih-Chieh Su 🔗 |
-
|
LoRA can Replace Time and Class Embeddings in Diffusion Probabilistic Models ( Poster ) > link | Joo Young Choi · Jaesung Park · Inkyu Park · Jaewoong Cho · Albert No · Ernest Ryu 🔗 |
-
|
Importance-Guided Diffusion ( Poster ) > link | Paris Flood · Pietro Lió 🔗 |
-
|
Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance ( Poster ) > link | Niket Patel · Luis Salamanca · Luis Barba 🔗 |
-
|
Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models ( Poster ) > link | Gabriele Corso · Yilun Xu · Valentin De Bortoli · Regina Barzilay · Tommi Jaakkola 🔗 |
-
|
Discrete Diffusion Language Modeling by Estimating the Ratios of the Data Distribution ( Poster ) > link | Aaron Lou · Chenlin Meng · Stefano Ermon 🔗 |
-
|
Adversarial Estimation of Topological Dimension with Harmonic Score Maps ( Poster ) > link | Eric Yeats · Cameron Darwin · Frank Liu · Hai Li 🔗 |
-
|
Generative Fractional Diffusion Models ( Poster ) > link |
11 presentersGabriel Nobis · Marco Aversa · Maximilian Springenberg · Michael Detzel · Stefano Ermon · Shinichi Nakajima · Roderick Murray-Smith · Sebastian Lapuschkin · Christoph Knochenhauer · Luis Oala · Wojciech Samek |
-
|
Text-Aware Diffusion Policies ( Poster ) > link | Calvin Luo · Chen Sun 🔗 |
-
|
CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images ( Poster ) > link | Aaron Gokaslan · A. Feder Cooper · Jasmine Collins · Landan Seguin · Austin Jacobson · Mihir Patel · Jonathan Frankle · Cory Stephenson · Volodymyr Kuleshov 🔗 |
-
|
Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model ( Poster ) > link | Yida Chen · Fernanda Viégas · Martin Wattenberg 🔗 |
-
|
DiffusionShield: A Watermark for Data Copyright Protection against Generative Diffusion Models ( Poster ) > link | Yingqian Cui · Jie Ren · Han Xu · Pengfei He · Hui Liu · Lichao Sun · Yue XING · Jiliang Tang 🔗 |
-
|
Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models ( Poster ) > link | Song Mei · Yuchen Wu 🔗 |
-
|
Drag-guided diffusion models for vehicle image generation ( Poster ) > link | Nikos Arechiga · Frank Permenter · Binyang Song · Chenyang Yuan 🔗 |
-
|
Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps ( Poster ) > link | Miguel Espinosa Minano · Elliot Crowley 🔗 |
-
|
DFU: scale-robust diffusion model for zero-shot super-resolution image generation ( Poster ) > link | Alexander Havrilla · Kevin Rojas · Wenjing Liao · Molei Tao 🔗 |
-
|
Diffusion Models without Attention ( Poster ) > link | Nathan Yan · Jiatao Gu · Alexander Rush 🔗 |
-
|
Effective Data Augmentation With Diffusion Models ( Poster ) > link | Brandon Trabucco · Kyle Doherty · Max Gurinas · Russ Salakhutdinov 🔗 |
-
|
DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting ( Poster ) > link | Salva Rühling Cachay · Bo Zhao · Hailey Joren · Rose Yu 🔗 |
-
|
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models ( Poster ) > link | Maitreya Patel · Tejas Gokhale · Chitta Baral · 'YZ' Yezhou Yang 🔗 |
-
|
Functional Flow Matching ( Poster ) > link | Gavin Kerrigan · Giosue Migliorini · Padhraic Smyth 🔗 |
-
|
Diffusion-Augmented Neural Processes ( Poster ) > link | Lorenzo Bonito · James Requeima · Aliaksandra Shysheya · Richard Turner 🔗 |
-
|
Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks ( Poster ) > link | Luca Scimeca · Alexander Rubinstein · Armand Nicolicioiu · Damien Teney · Yoshua Bengio 🔗 |
-
|
Score Normalization for a Faster Diffusion Exponential Integrator Sampler (DEIS) ( Poster ) > link | Guoxuan Xia · Duolikun Danier · Ayan Das · Stathi Fotiadis · Farhang Nabiei · Ushnish Sengupta · Alberto Bernacchia 🔗 |
-
|
AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model ( Poster ) > link | Zibin Dong · Yifu Yuan · Jianye Hao · Fei Ni · Yao Mu · YAN ZHENG · Yujing Hu · Tangjie Lv · Changjie Fan · ZHIPENG HU 🔗 |
-
|
LC-SD: Realistic Endoscopic Image Generation with Stable Diffusion and ControlNet ( Poster ) > link | Joanna Kaleta · Diego Dall'Alba · Szymon Plotka · Przemyslaw Korzeniowski 🔗 |
-
|
Faster Training of Diffusion Models and Improved Density Estimation via Parallel Score Matching ( Poster ) > link | Etrit Haxholli · Marco Lorenzi 🔗 |
-
|
Masked Multi-time Diffusion for Multi-modal Generative Modeling ( Poster ) > link | Mustapha BOUNOUA · Giulio Franzese · Pietro Michiardi 🔗 |
-
|
MOFDiff: Coarse-grained Diffusion for Metal-Organic Framework Design ( Poster ) > link | Xiang Fu · Tian Xie · Andrew Rosen · Tommi Jaakkola · Jake Smith 🔗 |
-
|
Successfully Applying Lottery Ticket Hypothesis to Diffusion Model ( Poster ) > link | Chao Jiang · Bo Hui · Bohan Liu · Da Yan 🔗 |
-
|
Strong generalization in diffusion models ( Poster ) > link | Zahra Kadkhodaie · Florentin Guth · Eero Simoncelli · Stephane Mallat 🔗 |
-
|
Diffusion Models With Learned Adaptive Noise Processes ( Poster ) > link | Subham Sahoo · Aaron Gokaslan · Christopher De Sa · Volodymyr Kuleshov 🔗 |
-
|
Generalized Contrastive Divergence: Joint Training of Energy-Based Model and Diffusion Model through Inverse Reinforcement Learning ( Poster ) > link | Sangwoong Yoon · Dohyun Kwon · Himchan Hwang · Yung-Kyun Noh · Frank Park 🔗 |
-
|
Diffusing More Objects for Semi-Supervised Domain Adaptation with Less Labeling ( Poster ) > link | Leander van den Heuvel · Gertjan Burghouts · David Zhang · Gwenn Englebienne · Sabina van Rooij 🔗 |
-
|
Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation ( Poster ) > link | Giorgio Giannone · Akash Srivastava · Ole Winther · Faez Ahmed 🔗 |
-
|
TADA: Timestep-Aware Data Augmentation for Diffusion Models ( Poster ) > link | NaHyeon Park · Kunhee Kim · Song Park · Jung-Woo Ha · Hyunjung Shim 🔗 |
-
|
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints ( Poster ) > link | Jian Chen · Ruiyi Zhang · Yufan Zhou · Rajiv Jain · Zhiqiang Xu · Ryan Rossi · Changyou Chen 🔗 |
-
|
Manifold Diffusion Fields ( Poster ) > link | Ahmed Elhag · Yuyang Wang · Joshua Susskind · Miguel Angel Bautista 🔗 |
-
|
Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models ( Poster ) > link | Hyeonho Jeong · Jong Chul Ye 🔗 |