Workshop
Human Evaluation of Generative Models
Divyansh Kaushik · Jennifer Hsia · Jessica Huynh · Yonadav Shavit · Samuel Bowman · Ting-Hao Huang · Douwe Kiela · Zachary Lipton · Eric Michael Smith
Room 290
Sat 3 Dec, 7:30 a.m. PST
Chat is not available.
Timezone: America/Los_Angeles
Schedule
|
Sat 7:30 a.m. - 7:45 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
SlidesLive Video |
Divyansh Kaushik 🔗 |
|
Sat 7:45 a.m. - 8:15 a.m.
|
Invited Keynote by Jason Weston
(
Keynote
)
>
SlidesLive Video |
Jason Weston 🔗 |
|
Sat 8:15 a.m. - 8:25 a.m.
|
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets
(
Oral
)
>
link
SlidesLive Video |
Philippe Laban · Chien-Sheng Wu · Wenhao Liu · Caiming Xiong 🔗 |
|
Sat 8:25 a.m. - 8:35 a.m.
|
Are GAN Biased? Evaluating GAN-Generated Facial Images via Crowdsourcing
(
Oral
)
>
link
SlidesLive Video |
Hangzhi Guo · Lizhen Zhu · Ting-Hao Huang 🔗 |
|
Sat 8:35 a.m. - 8:45 a.m.
|
Towards Credible Human Evaluation of Open-Domain Dialog Systems Using Interactive Setup
(
Oral
)
>
link
SlidesLive Video |
Sijia Liu · Patrick Lange · Behnam Hedayatnia · Alexandros Papangelis · Di Jin · Andrew Wirth · Yang Liu · Dilek Hakkani-Tur 🔗 |
|
Sat 8:45 a.m. - 9:30 a.m.
|
Panel on Technical Challenges Associated with Reliable Human Evaluations of Generative Models
(
Discussion Panel
)
>
|
Long Ouyang · Tongshuang Wu · Zachary Lipton 🔗 |
|
Sat 9:30 a.m. - 11:00 a.m.
|
Lunch Break
|
🔗 |
|
Sat 11:00 a.m. - 11:50 a.m.
|
Discussion on Policy Challenges Associated with Generative Models
(
Discussion Panel
)
>
|
Irene Solaiman · Russell Wald · Yonadav Shavit · Long Ouyang 🔗 |
|
Sat 11:50 a.m. - 12:00 p.m.
|
Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark
(
Oral
)
>
link
SlidesLive Video |
14 presentersVitali Petsiuk · Alexander E. Siemenn · Saisamrit Surbehera · Qi Qi Chin · Keith Tyser · Gregory Hunter · Arvind Raghavan · Yann Hicke · Bryan Plummer · Ori Kerret · Tonio Buonassisi · Kate Saenko · Armando Solar-Lezama · Iddo Drori |
|
Sat 12:00 p.m. - 12:10 p.m.
|
Can There be Art Without an Artist?
(
Oral
)
>
link
SlidesLive Video |
Avijit Ghosh · Genoveva Fossas 🔗 |
|
Sat 12:10 p.m. - 12:20 p.m.
|
Best Prompts for Text-to-Image Models and How to Find Them
(
Oral
)
>
link
SlidesLive Video |
Nikita Pavlichenko · Fedor Zhdanov · Dmitry Ustalov 🔗 |
|
Sat 12:20 p.m. - 12:30 p.m.
|
Evaluation of Synthetic Datasets for Conversational Recommender Systems
(
Oral
)
>
link
SlidesLive Video |
Harsh Lara · Manoj Tiwari 🔗 |
|
Sat 12:30 p.m. - 12:45 p.m.
|
Coffee Break
|
🔗 |
|
Sat 12:45 p.m. - 1:35 p.m.
|
Panel and QnA with Science Funders Interested in Reliable Human Evaluation of Generative Models
(
Panel
)
>
SlidesLive Video |
Brittany Smith · Eric Sears · Yonadav Shavit 🔗 |
|
Sat 1:35 p.m. - 1:45 p.m.
|
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
(
Oral
)
>
link
SlidesLive Video |
Vikas Raunak · Matt Post · Arul Menezes 🔗 |
|
Sat 1:45 p.m. - 1:55 p.m.
|
Sensemaking Interfaces for Human Evaluation of Language Model Outputs
(
Oral
)
>
link
SlidesLive Video |
Katy Gero · Jonathan Kummerfeld · Elena Glassman 🔗 |
|
Sat 1:55 p.m. - 2:05 p.m.
|
The Reasonable Effectiveness of Diverse Evaluation Data
(
Oral
)
>
link
SlidesLive Video |
Lora Aroyo · Mark Diaz · Christopher M. Homan · Vinodkumar Prabhakaran · Alex Taylor · Ding Wang 🔗 |
|
Sat 2:05 p.m. - 2:15 p.m.
|
Closing Remarks
(
Closing Remarks
)
>
SlidesLive Video |
Jessica Huynh 🔗 |