Skip to yearly menu bar Skip to main content


Poster

Membership Inference Attacks against Large Vision-Language Models

Zhan Li · Yongtao Wu · Yihang Chen · Francesco Tonin · Elias Abad Rocamora · Volkan Cevher

[ ]
Wed 11 Dec 4:30 p.m. PST — 7:30 p.m. PST

Abstract:

Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records in their training data. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue due to the absence of a standardized dataset and suitable methodology. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new detection metric called MaxRényi-K\%, which is based on the confidence of model outputs and applies to both text and image data. We believe our work can enhance the understanding and methodology of MIA in the context of VLLMs.

Live content is unavailable. Log in and register to view live content