Workshop
2nd Workshop on Self-Supervised Learning: Theory and Practice
Pengtao Xie · Ishan Misra · Pulkit Agrawal · Abdelrahman Mohamed · Shentong Mo · Youwei Liang · Jeannette Bohg · Kristina N Toutanova
Tue 14 Dec, 7 a.m. PST
Self-supervised learning (SSL) is an unsupervised approach for representation learning without relying on human-provided labels. It creates auxiliary tasks on unlabeled input data and learns representations by solving these tasks. SSL has demonstrated great success on images (e.g., MoCo [19], PIRL [9], SimCLR [20]) and texts (e.g., BERT [21]) and has shown promising results in other data modalities, including graphs, time-series, audio, etc. On a wide variety of tasks, SSL without using human-provided labels achieves performance that is close to fully supervised approaches. The existing SSL research mostly focuses on improving the empirical performance without a theoretical foundation. While the proposed SSL approaches are empirically effective, theoretically why they perform well is not clear. For example, why certain auxiliary tasks in SSL perform better than others? How many unlabeled data examples are needed by SSL to learn a good representation? How is the performance of SSL affected by neural architectures? In this workshop, we aim to bridge this gap between theory and practice. We bring together SSL-interested researchers from various domains to discuss the theoretical foundations of empirically well-performing SSL approaches and how the theoretical insights can further improve SSL’s empirical performance. Different from previous SSL-related workshops which focus on empirical effectiveness of SSL approaches without considering their theoretical foundations, our workshop focuses on establishing the theoretical foundation of SSL and providing theoretical insights for developing new SSL approaches.
Schedule
Tue 7:00 a.m. - 7:10 a.m.
|
Opening remarks
(
Opening
)
>
SlidesLive Video |
🔗 |
Tue 7:10 a.m. - 7:40 a.m.
|
Invited talk 1
(
Talk
)
>
SlidesLive Video |
Stefanie Jegelka 🔗 |
Tue 7:40 a.m. - 8:10 a.m.
|
Invited talk 2
(
Talk
)
>
SlidesLive Video |
Sanjeev Arora 🔗 |
Tue 8:10 a.m. - 8:40 a.m.
|
Invited talk 3
(
Talk
)
>
SlidesLive Video |
Jie Tang 🔗 |
Tue 8:40 a.m. - 9:40 a.m.
|
Poster session I
(
Poster Session
)
>
|
🔗 |
Tue 9:40 a.m. - 9:50 a.m.
|
Contributed talk 1
(
Talk
)
>
SlidesLive Video |
Guojun Zhang 🔗 |
Tue 9:50 a.m. - 10:00 a.m.
|
Contributed talk 2
(
Talk
)
>
SlidesLive Video |
Ching-Yun Ko 🔗 |
Tue 10:00 a.m. - 10:30 a.m.
|
Invited talk 4
(
Talk
)
>
SlidesLive Video |
Tengyu Ma 🔗 |
Tue 10:30 a.m. - 11:00 a.m.
|
Invited talk 5
(
Talk
)
>
SlidesLive Video |
Ian Fischer 🔗 |
Tue 11:00 a.m. - 12:00 p.m.
|
Lunch break
|
🔗 |
Tue 12:00 p.m. - 12:30 p.m.
|
Invited talk 6
(
Talk
)
>
SlidesLive Video |
Louis-Philippe Morency 🔗 |
Tue 12:30 p.m. - 1:00 p.m.
|
Invited talk 7
(
Talk
)
>
SlidesLive Video |
Jason Lee 🔗 |
Tue 1:00 p.m. - 1:30 p.m.
|
Invited talk 8
(
Talk
)
>
SlidesLive Video |
Mihaela van der Schaar 🔗 |
Tue 1:30 p.m. - 2:30 p.m.
|
Poster session II
(
Poster Session
)
>
|
🔗 |
Tue 2:30 p.m. - 2:40 p.m.
|
Contributed talk 3
(
Talk
)
>
SlidesLive Video |
Mehdi Azabou 🔗 |
Tue 2:40 p.m. - 2:50 p.m.
|
Contributed talk 4
(
Talk
)
>
|
Fangyu Liu 🔗 |
Tue 2:50 p.m. - 3:00 p.m.
|
Contributed talk 5
(
Talk
)
>
SlidesLive Video |
Menglin YANG 🔗 |
Tue 3:00 p.m. - 3:30 p.m.
|
Invited talk 9
(
Talk
)
>
SlidesLive Video |
Samuel Bowman 🔗 |
Tue 3:30 p.m. - 3:35 p.m.
|
Closing remarks
(
Closing remarks
)
>
SlidesLive Video |
🔗 |