Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Workshop on Behavioral Machine Learning

Unexploited Information Value in Human-AI Collaboration

Ziyang Guo · Yifan Wu · Jason Hartline · Jessica Hullman


Abstract:

Humans and AIs are often paired on decision tasks with the expectation of achievingcomplementary performance -- where the combination of human and AI outper-forms either one alone. However, how to improve performance of a human-AIteam is often not clear without knowing more about what particular informationand strategies each agent employs. In this paper, we propose a model based instatistically decision theory to analyze human-AI collaboration from the perspec-tive of what information could be used to improve a human or AI decision. Wedemonstrate our model on a deepfake detection task to investigate seven video-levelfeatures by their unexploited value of information. We compare the human alone,AI alone and human-AI team and offer insights on how the AI assistance impactspeople’s usage of the information and what information that the AI exploits wellmight be useful for improving human decisions.

Chat is not available.