Workshop
Gaze Meets ML
Amarachi Blessing Mbakwe · Joy T Wu · Dario Zanca · Elizabeth Krupinski · Satyananda Kashyap · Alexandros Karargyris
Room 240 - 241
Sat 16 Dec, 6:15 a.m. PST
Eye gaze has proven to be a cost-efficient way to collect large-scale physiological data that can reveal the underlying human attentional patterns in real-life workflows, and thus has long been explored as a signal to directly measure human-related cognition in various domains. Physiological data (including but not limited to eye gaze) offer new perception capabilities, which could be used in several ML domains, e.g., egocentric perception, embodied AI, NLP, etc. They can help infer human perception, intentions, beliefs, goals, and other cognition properties that are much needed for human-AI interactions and agent coordination. In addition, large collections of eye-tracking data have enabled data-driven modeling of human visual attention mechanisms, both for saliency or scan path prediction, with twofold advantages: from the neuro-scientific perspective to understand biological mechanisms better, and from the AI perspective to equip agents with the ability to mimic or predict human behavior and improve interpretability and interactions.
The Gaze meets ML workshop aims at bringing together an active research community to collectively drive progress in defining and addressing core problems in gaze-assisted machine learning. This year the workshop will run its 2nd edition at NeurIPS again and it attracts a diverse group of researchers from academia and industry presenting novel works in this area of research.
Schedule
Sat 6:15 a.m. - 6:30 a.m.
|
Open Remarks
(
Opening Remarks
)
>
|
🔗 |
Sat 6:30 a.m. - 7:15 a.m.
|
Bertram Emil SHI (HKUST)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Sat 7:15 a.m. - 8:00 a.m.
|
Vidhya Navalpakkam (Google) - Accelerating human attention research via smartphones
(
Invited Talk
)
>
SlidesLive Video |
Vidhya Navalpakkam 🔗 |
Sat 8:00 a.m. - 8:30 a.m.
|
Coffee Break
|
🔗 |
Sat 8:00 a.m. - 8:30 a.m.
|
Poster Session
(
Poster Session
)
>
|
🔗 |
Sat 8:30 a.m. - 8:45 a.m.
|
Interaction-aware Dynamic 3D Gaze Estimation in Videos
(
Oral
)
>
link
SlidesLive Video |
Chenyi Kuang · Jeffrey O Kephart · Qiang Ji 🔗 |
Sat 8:45 a.m. - 9:00 a.m.
|
SuperVision: Self-Supervised Super-Resolution for Appearance-Based Gaze Estimation
(
Oral
)
>
link
SlidesLive Video |
Galen O'Shea · Majid Komeili 🔗 |
Sat 9:00 a.m. - 9:15 a.m.
|
EG-SIF: Improving Appearance Based Gaze Estimation using Self Improving Features
(
Oral
)
>
link
SlidesLive Video |
Vasudev Singh · Chaitanya Langde · Sourav Lakhotia · Vignesh Kannan · Shuaib Ahmed 🔗 |
Sat 9:15 a.m. - 9:30 a.m.
|
Planning by Active Sensing
(
Oral
)
>
link
SlidesLive Video |
Kaushik Lakshminarasimhan · Seren Zhu · Dora Angelaki 🔗 |
Sat 9:30 a.m. - 9:45 a.m.
|
Crafting Good Views of Medical Images for Contrastive Learning via Expert-level Visual Attention
(
Oral
)
>
link
SlidesLive Video |
Sheng Wang · Zihao Zhao · Lichi Zhang · Dinggang Shen · Qian Wang 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Evaluating Peripheral Vision as an Input Transformation to Understand Object Detection Model Behavior ( Poster ) > link | Anne Harrington · Vasha DuTell · Mark Hamilton · Ayush Tewari · Simon Stent · Bill Freeman · Ruth Rosenholtz 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Leveraging Multi-Modal Saliency and Fusion for Gaze Target Detection ( Poster ) > link | Athul Mathew · Arshad Ali Khan · Thariq Khalid · Faroq AL-Tam · Riad Souissi 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Exploring Foveation and Saccade for Improved Weakly-Supervised Localization ( Poster ) > link | Timur Ibrayev · Manish Nagaraj · Amitangshu Mukherjee · Kaushik Roy 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
SAM meets Gaze: Passive Eye Tracking for Prompt-based Instance Segmentation ( Poster ) > link | Daniel Beckmann · Jacqueline Kockwelp · Joerg Gromoll · Friedemann Kiefer · Benjamin Risse 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
GazeSAM: Interactive Image Segmentation with Eye Gaze and Segment Anything Model ( Poster ) > link | Bin Wang · Armstrong Aboah · Zheyuan Zhang · Hongyi Pan · Ulas Bagci 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
FoVAE: Reconstructive Foveation as a Self-Supervised Variational Inference Task for Visual Representation Learning ( Poster ) > link | Ivan Vegner · Siddharth N · Leonidas Doumas 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Human-like multiple object tracking through occlusion via gaze-following ( Poster ) > link | Benjamin Peters · Eivinas Butkus · Nikolaus Kriegeskorte 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Inverting cognitive models with machine learning to infer preferences from fixations ( Poster ) > link | Evan Russek · Frederick Callaway · Tom Griffiths 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Detection of Drowsiness and Impending Microsleep from Eye Movements ( Poster ) > link | Silvia Makowski · Paul Prasse · Lena A. Jäger · Tobias Scheffer 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
StatTexNet: Evaluating the Importance of Statistical Parameters for Pyramid-Based Texture and Peripheral Vision Models ( Poster ) > link | Christian Koevesdi · Vasha DuTell · Anne Harrington · Mark Hamilton · Bill Freeman · Ruth Rosenholtz 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Temporal Understanding of Gaze Communication with GazeTransformer ( Poster ) > link | Ryan Anthony de Belen · Gelareh Mohammadi · Arcot Sowmya 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Interaction-aware Dynamic 3D Gaze Estimation in Videos ( Poster ) > link | Chenyi Kuang · Jeffrey O Kephart · Qiang Ji 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
SuperVision: Self-Supervised Super-Resolution for Appearance-Based Gaze Estimation ( Poster ) > link | Galen O'Shea · Majid Komeili 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
An Attention-based Predictive Agent for Handwritten Numeral/Alphabet Recognition via Generation ( Poster ) > link | Bonny Banerjee · Murchana Baruah 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
EG-SIF: Improving Appearance Based Gaze Estimation using Self Improving Features ( Poster ) > link | Vasudev Singh · Chaitanya Langde · Sourav Lakhotia · Vignesh Kannan · Shuaib Ahmed 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Planning by Active Sensing ( Poster ) > link | Kaushik Lakshminarasimhan · Seren Zhu · Dora Angelaki 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Crafting Good Views of Medical Images for Contrastive Learning via Expert-level Visual Attention ( Poster ) > link | Sheng Wang · Zihao Zhao · Lichi Zhang · Dinggang Shen · Qian Wang 🔗 |
Sat 9:45 a.m. - 11:30 a.m.
|
Memory-Based Sequential Attention ( Poster ) > link | Jason Stock · Charles Anderson 🔗 |
Sat 10:00 a.m. - 11:30 a.m.
|
Lunch Break
|
🔗 |
Sat 11:30 a.m. - 12:15 p.m.
|
Tim Rolff (Universität Hamburg) - Gazing into the Crystal Ball: Predicting Future Gaze Events in Virtual Reality
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Sat 12:15 p.m. - 12:30 p.m.
|
Memory-Based Sequential Attention
(
Oral
)
>
link
SlidesLive Video |
Jason Stock · Charles Anderson 🔗 |
Sat 12:30 p.m. - 12:45 p.m.
|
An Attention-based Predictive Agent for Handwritten Numeral/Alphabet Recognition via Generation
(
Oral
)
>
link
SlidesLive Video |
Bonny Banerjee · Murchana Baruah 🔗 |
Sat 12:45 p.m. - 1:00 p.m.
|
Breakout Session (Instructions)
(
Break out session
)
>
|
🔗 |
Sat 1:00 p.m. - 1:30 p.m.
|
Coffee Break
|
🔗 |
Sat 1:00 p.m. - 2:30 p.m.
|
Breakout session
(
Breakout session
)
>
|
🔗 |
Sat 2:30 p.m. - 2:45 p.m.
|
Sponsors Talk and Award Ceremony
(
Sponsors
)
>
SlidesLive Video |
🔗 |
Sat 2:45 p.m. - 3:00 p.m.
|
Closing Remarks
(
Closing remarks
)
>
SlidesLive Video |
🔗 |