Workshop
OPT2020: Optimization for Machine Learning
Courtney Paquette · Mark Schmidt · Sebastian Stich · Quanquan Gu · Martin Takac
Fri 11 Dec, 3:15 a.m. PST
Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops.
Looking back over the past decade, a strong trend is apparent: The intersection of OPT and ML has grown to the point that now cutting-edge advances in optimization often arise from the ML community. The distinctive feature of optimization within ML is its departure from textbook approaches, in particular, its focus on a different set of goals driven by "big-data, nonconvexity, and high-dimensions," where both theory and implementation are crucial.
We wish to use OPT 2020 as a platform to foster discussion, discovery, and dissemination of the state-of-the-art in optimization as relevant to machine learning. And well beyond that: as a platform to identify new directions and challenges that will drive future research, and continue to build the OPT+ML joint research community.
Invited Speakers
Volkan Cevher (EPFL)
Michael Friedlander (UBC)
Donald Goldfarb (Columbia)
Andreas Krause (ETH, Zurich)
Suvrit Sra (MIT)
Rachel Ward (UT Austin)
Ashia Wilson (MSR)
Tong Zhang (HKUST)
Instructions
Please join us in gather.town for all breaks and poster sessions (Click "Open Link" on any break or poster session).
To see all submitted paper and posters, go to the "opt-ml website" at the top of the page.
Use RocketChat or Zoom link (top of page) if you want to ask the speaker a direct question during the Live Q&A and Contributed Talks.
Schedule
Fri 3:15 a.m. - 3:50 a.m.
|
Welcome event (gather.town) ( Social event/Break ) > link | Quanquan Gu · Courtney Paquette · Mark Schmidt · Sebastian Stich · Martin Takac 🔗 |
Fri 3:50 a.m. - 4:00 a.m.
|
Welcome remarks to Session 1
(
Opening remarks
)
>
|
Sebastian Stich 🔗 |
Fri 4:00 a.m. - 4:20 a.m.
|
Invited speaker: The Convexity of Learning Infinite-width Deep Neural Networks, Tong Zhang
(
Talk
)
>
SlidesLive Video |
Tong Zhang 🔗 |
Fri 4:20 a.m. - 4:30 a.m.
|
Live Q&A with Tong Zhang (Zoom)
(
Q&A
)
>
|
Sebastian Stich 🔗 |
Fri 4:30 a.m. - 4:50 a.m.
|
Invited speaker: Adaptation and universality in first-order methods, Volkan Cevher
(
Talk
)
>
|
Volkan Cevher 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed talks in Session 1 (Zoom) ( Multiple talks ) > link | Sebastian Stich · Laurent Condat · Zhize Li · Ohad Shamir · Tiffany Vlaar · Mohammadi Zaki 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Constraint-Based Regularization of Neural Networks, Tiffany Vlaar
(
Talk
)
>
link
SlidesLive Video |
Tiffany Vlaar 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?, Ohad Shamir ( Talk ) > link | Ohad Shamir 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Employing No Regret Learners for Pure Exploration in Linear Bandits, Mohammadi Zaki
(
Talk
)
>
link
SlidesLive Video |
Mohammadi Zaki 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Distributed Proximal Splitting Algorithms with Rates and Acceleration, Laurent Condat
(
Talk
)
>
link
SlidesLive Video |
Laurent Condat 🔗 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization, Zhize Li
(
Talk
)
>
link
SlidesLive Video |
Zhize Li 🔗 |
Fri 6:00 a.m. - 6:50 a.m.
|
Poster Session 1 (gather.town) ( Poster session ) > link |
26 presentersLaurent Condat · Tiffany Vlaar · Ohad Shamir · Mohammadi Zaki · Zhize Li · Guan-Horng Liu · Samuel Horváth · Mher Safaryan · Yoni Choukroun · Kumar Shridhar · Nabil Kahale · Jikai Jin · Pratik Kumar Jawanpuria · Gaurav Kumar Yadav · Kazuki Koyama · Junyoung Kim · Xiao Li · Saugata Purkayastha · Adil Salim · Dighanchal Banerjee · Peter Richtarik · Lakshman Mahto · Tian Ye · Bamdev Mishra · Huikang Liu · Jiajie Zhu |
Fri 6:50 a.m. - 7:00 a.m.
|
Welcome remarks to Session 2
(
Opening remarks
)
>
|
Martin Takac 🔗 |
Fri 7:00 a.m. - 7:20 a.m.
|
Invited speaker: Adaptive Sampling for Stochastic Risk-Averse Learning, Andreas Krause
(
Talk
)
>
SlidesLive Video |
Andreas Krause 🔗 |
Fri 7:20 a.m. - 7:30 a.m.
|
Live Q&A with Andreas Krause (Zoom)
(
Q&A
)
>
|
Martin Takac 🔗 |
Fri 7:30 a.m. - 7:50 a.m.
|
Invited speaker: Practical Kronecker-factored BFGS and L-BFGS methods for training deep neural networks, Donald Goldfarb
(
Talk
)
>
SlidesLive Video |
Donald Goldfarb 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed talks in Session 2 (Zoom)
(
Multiple talks
)
>
|
Martin Takac · Samuel Horváth · Guan-Horng Liu · Nicolas Loizou · Sharan Vaswani 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization, Samuel Horvath
(
Talk
)
>
link
SlidesLive Video |
Samuel Horváth 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence, Nicolas Loizou
(
Talk
)
>
link
SlidesLive Video |
Nicolas Loizou 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: DDPNOpt: Differential Dynamic Programming Neural Optimizer, Guan-Horng Liu
(
Talk
)
>
link
SlidesLive Video |
Guan-Horng Liu 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search), Sharan Vaswani
(
Talk
)
>
link
SlidesLive Video |
Sharan Vaswani 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: How to make your optimizer generalize better, Sharan Vaswani
(
Talk
)
>
link
SlidesLive Video |
Sharan Vaswani 🔗 |
Fri 8:30 a.m. - 9:00 a.m.
|
Break (gather.town) link | 🔗 |
Fri 9:00 a.m. - 9:20 a.m.
|
Invited speaker: SGD without replacement: optimal rate analysis and more, Suvrit Sra
(
Talk
)
>
SlidesLive Video |
Suvrit Sra 🔗 |
Fri 9:20 a.m. - 9:30 a.m.
|
Live Q&A with Suvrit Sra (Zoom)
(
Q&A
)
>
|
Martin Takac 🔗 |
Fri 9:45 a.m. - 10:50 a.m.
|
Poster Session 2 (gather.town) ( Poster session ) > link |
26 presentersSharan Vaswani · Nicolas Loizou · Wenjie Li · Preetum Nakkiran · Zhan Gao · Sina Baghal · Jingfeng Wu · Roozbeh Yousefzadeh · Jinyi Wang · Jing Wang · Cong Xie · Anastasia Borovykh · Stanislaw Jastrzebski · Soham Dan · Yiliang Zhang · Mark Tuddenham · Sarath Pattathil · Ievgen Redko · Jeremy Cohen · Yasaman Esfandiari · Zhanhong Jiang · Mostafa ElAraby · Chulhee Yun · Michael Psenka · Robert Gower · Xiaoyu Wang |
Fri 10:50 a.m. - 11:00 a.m.
|
Welcome remarks to Session 3
(
Opening remarks
)
>
|
Mark Schmidt 🔗 |
Fri 11:00 a.m. - 11:20 a.m.
|
Invited speaker: Stochastic Geodesic Optimization, Ashia Wilson
(
Talk
)
>
|
Ashia Wilson 🔗 |
Fri 11:20 a.m. - 11:30 a.m.
|
Live Q&A with Ashia Wilson (Zoom)
(
Q&A
)
>
|
Mark Schmidt 🔗 |
Fri 11:30 a.m. - 11:50 a.m.
|
Invited speaker: Concentration for matrix products, and convergence of Oja’s algorithm for streaming PCA, Rachel Ward
(
Talk
)
>
|
Rachel Ward 🔗 |
Fri 11:50 a.m. - 12:00 p.m.
|
Live Q&A with Rachel Ward (Zoom)
(
Q&A
)
>
|
Mark Schmidt 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed talks in Session 3 (Zoom)
(
Multiple talks
)
>
|
Mark Schmidt · Zhan Gao · Wenjie Li · Preetum Nakkiran · Denny Wu · Chengrun Yang 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Variance Reduction on Adaptive Stochastic Mirror Descent, Wenjie Li
(
Talk
)
>
link
SlidesLive Video |
Wenjie Li 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems, Preetum Nakkiran
(
Talk
)
>
link
SlidesLive Video |
Preetum Nakkiran 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: When Does Preconditioning Help or Hurt Generalization?, Denny Wu
(
Talk
)
>
link
SlidesLive Video |
Denny Wu 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Incremental Greedy BFGS: An Incremental Quasi-Newton Method with Explicit Superlinear Rate, Zhan Gao
(
Talk
)
>
link
SlidesLive Video |
Zhan Gao 🔗 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: TenIPS: Inverse Propensity Sampling for Tensor Completion, Chengrun Yang
(
Talk
)
>
link
SlidesLive Video |
Chengrun Yang 🔗 |
Fri 12:30 p.m. - 1:30 p.m.
|
Break (gather.town) link | 🔗 |
Fri 1:30 p.m. - 1:50 p.m.
|
Invited speaker: Fast convergence of stochastic subgradient method under interpolation, Michael Friedlander
(
Talk
)
>
SlidesLive Video |
Michael Friedlander 🔗 |
Fri 1:30 p.m. - 1:35 p.m.
|
Intro to Invited Speaker 8
(
Organizer intro
)
>
|
Mark Schmidt 🔗 |
Fri 1:50 p.m. - 2:00 p.m.
|
Live Q&A with Michael Friedlander (Zoom)
(
Q&A
)
>
|
Mark Schmidt 🔗 |
Fri 2:00 p.m. - 2:50 p.m.
|
Poster Session 3 (gather.town) ( Poster session ) > link |
23 presentersDenny Wu · Chengrun Yang · Tolga Ergen · sanae lotfi · Charles Guille-Escuret · Boris Ginsburg · Hanbake Lyu · Cong Xie · David Newton · Debraj Basu · Yewen Wang · James Lucas · MAOJIA LI · Lijun Ding · Jose Javier Gonzalez Ortiz · Reyhane Askari Hemmat · Zhiqi Bu · Neal Lawton · Kiran Thekumparampil · Jiaming Liang · Lindon Roberts · Jingyi Zhu · Dongruo Zhou |
Fri 2:50 p.m. - 3:00 p.m.
|
Welcome remarks to Session 4
(
Opening remarks
)
>
|
Quanquan Gu 🔗 |
Fri 3:00 p.m. - 3:20 p.m.
|
Invited speaker: Online nonnegative matrix factorization for Markovian and other real data, Deanna Needell and Hanbaek Lyu
(
Talk
)
>
SlidesLive Video |
Hanbake Lyu · Deanna Needell 🔗 |
Fri 3:20 p.m. - 3:30 p.m.
|
Live Q&A with Deanna Needell and Hanbake Lyu (Zoom)
(
Q&A
)
>
|
Quanquan Gu 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed talks in Session 4 (Zoom)
(
Multiple talks
)
>
|
Quanquan Gu · sanae lotfi · Charles Guille-Escuret · Tolga Ergen · Dongruo Zhou 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Stochastic Damped L-BFGS with controlled norm of the Hessian approximation, Sanae Lotfi
(
Talk
)
>
link
SlidesLive Video |
sanae lotfi 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: A Study of Condition Numbers for First-Order Optimization, Charles Guille-Escuret
(
Talk
)
>
link
SlidesLive Video |
Charles Guille-Escuret 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Affine-Invariant Analysis of Frank-Wolfe on Strongly Convex Sets, Lewis Liu
(
Talk
)
>
link
SlidesLive Video |
🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time, Tolga Ergen
(
Talk
)
>
link
SlidesLive Video |
Tolga Ergen 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization, Dongruo Zhou
(
Talk
)
>
link
SlidesLive Video |
Dongruo Zhou 🔗 |
Fri 4:00 p.m. - 4:05 p.m.
|
Closing remarks
(
Closing remarks
)
>
|
Quanquan Gu · Courtney Paquette · Mark Schmidt · Sebastian Stich · Martin Takac 🔗 |