Workshop
Self-Supervised Learning for Speech and Audio Processing
Abdelrahman Mohamed · Hung-yi Lee · Shinji Watanabe · Shang-Wen Li · Tara Sainath · Karen Livescu
Fri 11 Dec, 6:50 a.m. PST
There is a trend in the machine learning community to adopt self-supervised approaches to pre-train deep networks. Self-supervised learning utilizes proxy supervised learning tasks, for example, distinguishing parts of the input signal from distractors, or generating masked input segments conditioned on the unmasked ones, to obtain training data from unlabeled corpora. These approaches make it possible to use a tremendous amount of unlabeled data on the web to train large networks and solve complicated tasks. ELMo, BERT, and GPT in NLP are famous examples in this direction. Recently self-supervised approaches for speech and audio processing are also gaining attention. These approaches combine methods for utilizing no or partial labels, unpaired text and audio data, contextual text and video supervision, and signals from user interactions. Although the research direction of self-supervised learning is active in speech and audio processing, current works are limited to several problems such as automatic speech recognition, speaker identification, and speech translation, partially due to the diversity of modeling in various speech and audio processing problems. There is still much unexplored territory in the research direction for self-supervised learning.
This workshop will bring concentrated discussions on self-supervision for the field of speech and audio processing via several invited talks, oral and poster sessions with high-quality papers, and a panel of leading researchers from academia and industry. Alongside research work on new self-supervised methods, data, applications, and results, this workshop will call for novel work on understanding, analyzing, and comparing different self-supervision approaches for speech and audio processing. The workshop aims to:
- Review existing and inspire new self-supervised methods and results,
- Motivate the application of self-supervision approaches to more speech and audio processing problems in academia and industry, and encourage discussion amongst experts and practitioners from the two realms,
- Encourage works on studying methods for understanding learned representations, comparing different self-supervision methods and comparing self-supervision to other self-training as well as transfer learning methods that low-resource speech and audio processing have long utilized,
- Facilitate communication within the field of speech and audio processing (e.g., people who attend conferences such as INTERSPEECH and ICASSP) as well as between the field and the whole machine learning community for sharing knowledge, ideas, and data, and encourage future collaboration to inspire innovation in the field and the whole community.
Schedule
Fri 6:50 a.m. - 7:00 a.m.
|
Opening remarks
(
Introduction
)
>
|
Hung-yi Lee 🔗 |
Fri 7:00 a.m. - 7:35 a.m.
|
Invited talk - A Broad Perspective into Self Supervised Learning for Speech Recognition
(
Invited talk
)
>
|
Bhuvana Ramabhadran 🔗 |
Fri 7:35 a.m. - 7:45 a.m.
|
Q&A for invited talk - 1
(
Q&A
)
>
|
🔗 |
Fri 7:45 a.m. - 8:20 a.m.
|
Invited talk - Multimodal Distant Supervision
(
Invited talk
)
>
SlidesLive Video |
Mark Hasegawa-Johnson 🔗 |
Fri 8:20 a.m. - 8:30 a.m.
|
Q&A for invited talk - Multimodal Distant Supervision
(
Q&A
)
>
|
🔗 |
Fri 8:30 a.m. - 8:40 a.m.
|
Self-Supervised Learning using Contrastive Mixtures for Personalized Speech Enhancement
(
Contributed talk
)
>
SlidesLive Video |
Aswin Sivaraman 🔗 |
Fri 8:40 a.m. - 8:50 a.m.
|
Self-supervised Pre-training Reduces Label Permutation Instability of Speech Separation
(
Contributed talk
)
>
SlidesLive Video |
Sung-Feng Huang 🔗 |
Fri 8:50 a.m. - 9:00 a.m.
|
Augmentation adversarial training for self-supervised speaker recognition
(
Contributed talk
)
>
SlidesLive Video |
Jaesung Huh 🔗 |
Fri 9:00 a.m. - 9:10 a.m.
|
Neural Composition: Learning to Generate from Multiple Models
(
Contributed talk
)
>
SlidesLive Video |
Denis Filimonov 🔗 |
Fri 9:10 a.m. - 9:20 a.m.
|
Towards Semi-Supervised Semantics Understanding from Speech
(
Contributed talk
)
>
SlidesLive Video |
Cheng-I Jeff Lai 🔗 |
Fri 9:20 a.m. - 9:30 a.m.
|
The Zero Resource Speech Benchmark 2021. Metrics and baselines for unsupervised spoken language modeling
(
Contributed talk
)
>
SlidesLive Video |
Tu Anh Nguyen 🔗 |
Fri 9:30 a.m. - 9:45 a.m.
|
Q&A for contributed talks between 11:30 and 12:30
(
Q&A
)
>
|
🔗 |
Fri 9:45 a.m. - 10:00 a.m.
|
Break
|
🔗 |
Fri 10:00 a.m. - 10:35 a.m.
|
Invited talk - Speech Processing with Weak Supervision
(
Invited talk
)
>
SlidesLive Video |
Dong Yu 🔗 |
Fri 10:35 a.m. - 10:45 a.m.
|
Q&A for invited talk - Speech Processing with Weak Supervision
(
Q&A
)
>
|
🔗 |
Fri 10:45 a.m. - 10:55 a.m.
|
Towards Localisation of Keywords in Speech Using Weak Supervision
(
Contributed talk
)
>
SlidesLive Video |
Kayode Olaleye 🔗 |
Fri 10:55 a.m. - 11:05 a.m.
|
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
(
Contributed talk
)
>
SlidesLive Video |
Wei-Ning Hsu 🔗 |
Fri 11:05 a.m. - 11:15 a.m.
|
Self-Supervised Audio-Visual Separation of On-Screen Sounds from Unlabeled Videos
(
Contributed talk
)
>
SlidesLive Video |
Efthymios Tzinis 🔗 |
Fri 11:15 a.m. - 11:25 a.m.
|
Multi-Format Contrastive Learning of Audio Representations
(
Contributed talk
)
>
SlidesLive Video |
Aaron van den Oord 🔗 |
Fri 11:25 a.m. - 11:40 a.m.
|
Q&A for contributed talks between 1:45 and 2:25
(
Q&A
)
>
|
🔗 |
Fri 11:40 a.m. - 11:55 a.m.
|
Break
|
🔗 |
Fri 11:55 a.m. - 12:30 p.m.
|
Invited talk - Underfitting and Uncertainty in Self-Supervised Predictive Models
(
Invited talk
)
>
|
Chelsea Finn 🔗 |
Fri 12:30 p.m. - 12:40 p.m.
|
Q&A for invited talk - Underfitting and Uncertainty in Self-Supervised Predictive Models
(
Q&A
)
>
|
🔗 |
Fri 12:40 p.m. - 1:15 p.m.
|
Invited talk - Towards robust self-supervised learning of speech representations
(
Invited talk
)
>
SlidesLive Video |
Mirco Ravanelli 🔗 |
Fri 1:15 p.m. - 1:25 p.m.
|
Q&A for invited talk - Towards robust self-supervised learning of speech representations
(
Q&A
)
>
|
🔗 |
Fri 1:25 p.m. - 1:35 p.m.
|
Similarity Analysis of Self-Supervised Speech Representations
(
Contributed talk
)
>
SlidesLive Video |
Yu-An Chung 🔗 |
Fri 1:35 p.m. - 1:45 p.m.
|
Representation Learning for Sequence Data with Deep Autoencoding Predictive
(
Contributed talk
)
>
SlidesLive Video |
Junwen Bai 🔗 |
Fri 1:45 p.m. - 1:55 p.m.
|
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
(
Contributed talk
)
>
SlidesLive Video |
Yu Zhang 🔗 |
Fri 1:55 p.m. - 2:05 p.m.
|
A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embedding
(
Contributed talk
)
>
SlidesLive Video |
Puyuan Peng 🔗 |
Fri 2:05 p.m. - 2:15 p.m.
|
HUBERT: How much can a bad teacher benefit ASR pre-training?
(
Contributed talk
)
>
SlidesLive Video |
Wei-Ning Hsu 🔗 |
Fri 2:15 p.m. - 2:30 p.m.
|
Q&A for contributed talks between 4:25 and 5:15
(
Q&A
)
>
|
🔗 |
Fri 2:30 p.m. - 2:45 p.m.
|
Break
|
🔗 |
Fri 2:45 p.m. - 3:20 p.m.
|
Invited talk - Flexible contextualized speech representation learning for diverse downstream tasks
(
Invited talk
)
>
SlidesLive Video |
Katrin Kirchhhoff 🔗 |
Fri 3:20 p.m. - 3:30 p.m.
|
Q&A for invited talk - Flexible contextualized speech representation learning for diverse downstream tasks
(
Q&A
)
>
|
🔗 |
Fri 3:30 p.m. - 4:05 p.m.
|
Invited talk - De-noising Sequence-to-Sequence Pre-training
(
Invited talk
)
>
SlidesLive Video |
Luke Zettlemoyer 🔗 |
Fri 4:05 p.m. - 4:15 p.m.
|
Q&A for invited talk - De-noising Sequence-to-Sequence Pre-training
(
Q&A
)
>
|
🔗 |
Fri 4:15 p.m. - 4:25 p.m.
|
Closing remark
(
Introduction
)
>
|
Abdelrahman Mohamed 🔗 |