Datasets and Benchmarks
Dataset and Benchmark Poster Session 2
Joaquin Vanschoren · Serena Yeung
Moderator : Frank R Hutter
Virtual
The Datasets and Benchmarks track serves as a novel venue for high-quality publications, talks, and posters on highly valuable machine learning datasets and benchmarks, as well as a forum for discussions on how to improve dataset development. Datasets and benchmarks are crucial for the development of machine learning methods, but also require their own publishing and reviewing guidelines. For instance, datasets can often not be reviewed in a double-blind fashion, and hence full anonymization will not be required. On the other hand, they do require additional specific checks, such as a proper description of how the data was collected, whether they show intrinsic bias, and whether they will remain accessible.
Schedule
-
|
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
(
Poster
)
>
SlidesLive Video |
12 presentersSaahil Jain · Ashwin Agrawal · Adriel Saporta · Steven Truong · Du Nguyen Duong · Tan Bui · Pierre Chambon · Yuhao Zhang · Matthew Lungren · Andrew Ng · Curtis Langlotz · Pranav Rajpurkar |
-
|
One Million Scenes for Autonomous Driving: ONCE Dataset
(
Poster
)
>
SlidesLive Video |
13 presentersJiageng Mao · Niu Minzhe · ChenHan Jiang · hanxue liang · Jingheng Chen · Xiaodan Liang · Yamin Li · Chaoqiang Ye · Wei Zhang · Zhenguo Li · Jie Yu · Hang Xu · Chunjing XU |
-
|
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
(
Poster
)
>
SlidesLive Video |
15 presentersLinjie Li · Jie Lei · Zhe Gan · Licheng Yu · Yen-Chun Chen · Rohit Pillai · Yu Cheng · Luowei Zhou · Xin Wang · William Yang Wang · Tamara L Berg · Mohit Bansal · Jingjing Liu · Lijuan Wang · Zicheng Liu |
-
|
PASS: An ImageNet replacement for self-supervised pretraining without humans
(
Poster
)
>
SlidesLive Video |
Yuki Asano · Christian Rupprecht · Andrew Zisserman · Andrea Vedaldi 🔗 |
-
|
CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancer
(
Poster
)
>
link
SlidesLive Video |
Moein Sorkhei · Yue Liu · Hossein Azizpour · Edward Azavedo · Karin Dembrower · Dimitra Ntoula · Athanasios Zouzos · Fredrik Strand · Kevin Smith 🔗 |
-
|
PROCAT: Product Catalogue Dataset for Implicit Clustering, Permutation Learning and Structure Prediction
(
Poster
)
>
SlidesLive Video |
Mateusz Jurewicz · Leon Derczynski 🔗 |
-
|
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
(
Poster
)
>
SlidesLive Video |
12 presentersPaul Pu Liang · Yiwei Lyu · Xiang Fan · Zetian Wu · Yun Cheng · Jason Wu · Leslie (Yufan) Chen · Peter Wu · Michelle A. Lee · Yuke Zhu · Ruslan Salakhutdinov · Louis-Philippe Morency |
-
|
RedCaps: Web-curated image-text data created by the people, for the people
(
Poster
)
>
link
SlidesLive Video |
Karan Desai · Gaurav Kaul · Zubin Aysola · Justin Johnson 🔗 |
-
|
The PAIR-R24M Dataset for Multi-animal 3D Pose Estimation
(
Poster
)
>
link
SlidesLive Video |
Jesse Marshall · Ugne Klibaite · amanda gellis · Diego Aldarondo · Bence Olveczky · Timothy W Dunn 🔗 |
-
|
EventNarrative: A Large-scale Event-centric Dataset for Knowledge Graph-to-Text Generation
(
Poster
)
>
link
SlidesLive Video |
Anthony Colas · Ali Sadeghian · Yue Wang · Daisy Zhe Wang 🔗 |
-
|
ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data
(
Poster
)
>
link
SlidesLive Video |
11 presentersGilad Baruch · Zhuoyuan Chen · Afshin Dehghan · Yuri Feigin · Peter Fu · Thomas Gebauer · Daniel Kurz · Tal Dimry · Brandon Joffe · Arik Schwartz · Elad Shulman |
-
|
ImageNet-21K Pretraining for the Masses
(
Poster
)
>
SlidesLive Video |
Tal Ridnik · Emanuel Ben-Baruch · Asaf Noy · Lihi Zelnik 🔗 |
-
|
STAR: A Benchmark for Situated Reasoning in Real-World Videos
(
Poster
)
>
link
SlidesLive Video |
Bo Wu · Shoubin Yu · Zhenfang Chen · Josh Tenenbaum · Chuang Gan 🔗 |
-
|
Benchmarking Multimodal AutoML for Tabular Data with Text Fields
(
Poster
)
>
SlidesLive Video |
Xingjian Shi · Jonas Mueller · Nick Erickson · Mu Li · Alexander Smola 🔗 |
-
|
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
(
Poster
)
>
link
SlidesLive Video |
Jiyang Qi · Yan Gao · Yao Hu · Xinggang Wang · Xiaoyu Liu · Xiang Bai · Serge Belongie · Alan Yuille · Philip Torr · Song Bai 🔗 |
-
|
Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection
(
Poster
)
>
link
SlidesLive Video |
John Lambert · James Hays 🔗 |
-
|
Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting
(
Poster
)
>
link
SlidesLive Video |
13 presentersBenjamin Wilson · William Qi · Tanmay Agarwal · John Lambert · Jagjeet Singh · Siddhesh Khandelwal · Bowen Pan · Ratnesh Kumar · Andrew Hartnett · Jhony Kaesemodel Pontes · Deva Ramanan · Peter Carr · James Hays |
-
|
Constructing a Visual Dataset to Study the Effects of Spatial Apartheid in South Africa
(
Poster
)
>
SlidesLive Video |
Raesetje Sefala · Timnit Gebru · Luzango Mfupe · Nyalleng Moorosi · Richard Klein 🔗 |
-
|
The CLEAR Benchmark: Continual LEArning on Real-World Imagery
(
Poster
)
>
link
SlidesLive Video |
Zhiqiu Lin · Jia Shi · Deepak Pathak · Deva Ramanan 🔗 |
-
|
STEP: Segmenting and Tracking Every Pixel
(
Poster
)
>
SlidesLive Video |
13 presentersMark Weber · Jun Xie · Maxwell Collins · Yukun Zhu · Paul Voigtlaender · Hartwig Adam · Bradley Green · Andreas Geiger · Bastian Leibe · Daniel Cremers · Aljosa Osep · Laura Leal-Taixé · Liang-Chieh Chen |
-
|
What Ails One-Shot Image Segmentation: A Data Perspective
(
Poster
)
>
SlidesLive Video |
Mayur Hemani · Abhinav Patel · Tejas Shimpi · Anirudha Ramesh · Balaji Krishnamurthy 🔗 |
-
|
Pl@ntNet-300K: a plant image dataset with high label ambiguity and a long-tailed distribution
(
Poster
)
>
SlidesLive Video |
Camille Garcin · alexis joly · Pierre Bonnet · Antoine Affouard · Jean-Christophe Lombardo · Mathias Chouet · Maximilien Servajean · Titouan Lorieul · Joseph Salmon 🔗 |
-
|
ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation
(
Poster
)
>
SlidesLive Video |
Laurynas Karazija · Iro Laina · Christian Rupprecht 🔗 |
-
|
CropHarvest: A global dataset for crop-type classification
(
Poster
)
>
link
SlidesLive Video |
Gabriel Tseng · Ivan Zvonkov · Catherine Nakalembe · Hannah Kerner 🔗 |
-
|
RELLISUR: A Real Low-Light Image Super-Resolution Dataset
(
Poster
)
>
SlidesLive Video |
Andreas Aakerberg · Kamal Nasrollahi · Thomas Moeslund 🔗 |
-
|
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
(
Poster
)
>
link
SlidesLive Video |
13 presentersSanthosh Kumar Ramakrishnan · Aaron Gokaslan · Erik Wijmans · Oleksandr Maksymets · Alexander Clegg · John Turner · Eric Undersander · Wojciech Galuba · Andrew Westbury · Angel Chang · Manolis Savva · Yili Zhao · Dhruv Batra |
-
|
Intelligent Sight and Sound: A Chronic Cancer Facial Pain Dataset
(
Poster
)
>
SlidesLive Video |
Catherine Ordun · Alexandra Cha · Edward Raff · Byron Gaskin · Alexander Hanson · Mason Rule · Sanjay Purushotham · James Gulley 🔗 |
-
|
VFP290K: A Large-Scale Benchmark Dataset for Vision-based Fallen Person Detection
(
Poster
)
>
link
SlidesLive Video |
Jaeju An · Jeongho Kim · Hanbeen Lee · Jinbeom Kim · Junhyung Kang · Minha Kim · Saebyeol Shin · Minha Kim · Donghee Hong · Simon Woo 🔗 |
-
|
SKM-TEA: A Dataset for Accelerated MRI Reconstruction with Dense Image Labels for Quantitative Clinical Evaluation
(
Poster
)
>
link
SlidesLive Video |
12 presentersArjun Desai · Andrew Schmidt · Elka Rubin · Christopher Sandino · Marianne Black · Valentina Mazzoli · Kathryn Stevens · Robert Boutin · Christopher Ré · Garry Gold · Brian Hargreaves · Akshay Chaudhari |
-
|
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
(
Poster
)
>
SlidesLive Video |
Pan Lu · Liang Qiu · Jiaqi Chen · Tanglin Xia · Yizhou Zhao · Wei Zhang · Zhou Yu · Xiaodan Liang · Song-Chun Zhu 🔗 |
-
|
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
(
Poster
)
>
link
SlidesLive Video |
Hasam Khalid · Shahroz Tariq · Minha Kim · Simon Woo 🔗 |
-
|
Seasons in Drift: A Long Term Thermal Imaging Dataset for Studying Concept Drift
(
Poster
)
>
SlidesLive Video |
Ivan Nikolov · Mark Philip Philipsen · Jinsong Liu · Jacob Dueholm · Anders Johansen · Kamal Nasrollahi · Thomas Moeslund 🔗 |
-
|
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving
(
Poster
)
>
SlidesLive Video |
11 presentersJianhua Han · Xiwen Liang · Hang Xu · Kai Chen · Lanqing Hong · Jiageng Mao · Chaoqiang Ye · Wei Zhang · Zhenguo Li · Xiaodan Liang · Chunjing XU |
-
|
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation
(
Poster
)
>
SlidesLive Video |
Junjue Wang · Zhuo Zheng · Ailong Ma · Xiaoyan Lu · Yanfei Zhong 🔗 |
-
|
A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer
(
Poster
)
>
SlidesLive Video |
威佳 吴 · Debing Zhang · Yuanqiang Cai · Sibo Wang · Jiahong Li · Zhuang Li · Yejun Tang · Hong Zhou 🔗 |
-
|
DENETHOR: The DynamicEarthNET dataset for Harmonized, inter-Operable, analysis-Ready, daily crop monitoring from space
(
Poster
)
>
SlidesLive Video |
12 presentersLukas Kondmann · Aysim Toker · Marc Rußwurm · Andrés Camero · Devis Peressuti · Grega Milcinski · Pierre-Philippe Mathieu · Nicolas Longepe · Timothy Davis · Giovanni Marchisio · Laura Leal-Taixé · Xiaoxiang Zhu |
-
|
A realistic approach to generate masked faces applied on two novel masked face recognition data sets
(
Poster
)
>
SlidesLive Video |
Tudor-Alexandru Mare · Georgian Duta · Iuliana Georgescu · Adrian Sandru · Bogdan Alexe · Marius Popescu · Radu Tudor Ionescu 🔗 |
-
|
AP-10K: A Benchmark for Animal Pose Estimation in the Wild
(
Poster
)
>
SlidesLive Video |
Hang Yu · Yufei Xu · Jing Zhang · Wei Zhao · Ziyu Guan · Dacheng Tao 🔗 |
-
|
The Met Dataset: Instance-level Recognition for Artworks
(
Poster
)
>
link
SlidesLive Video |
Nikolaos-Antonios Ypsilantis · Noa Garcia · Guangxing Han · Sarah Ibrahimi · Nanne van Noord · Giorgos Tolias 🔗 |
-
|
WikiChurches: A Fine-Grained Dataset of Architectural Styles with Real-World Challenges
(
Poster
)
>
SlidesLive Video |
Björn Barz · Joachim Denzler 🔗 |
-
|
Benchmarks for Corruption Invariant Person Re-identification
(
Poster
)
>
SlidesLive Video |
Minghui Chen · Zhiqiang Wang · Feng Zheng 🔗 |