Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Attributing Model Behavior at Scale (ATTRIB)

How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation Threshold

Sahil Verma · Royi Rassin · Arnav Das · Gantavya Bhatt · Preethi Seshadri · Chirag Shah · Jeff A Bilmes · Hannaneh Hajishirzi · Yanai Elazar


Abstract:

Text-to-image models are trained using large datasets collected by scraping image-text pairs from the internet. These datasets often include private, copyrighted, and licensed material. Training models on such datasets enables them to generate images with such content, which might violate copyright laws and individual privacy. This phenomenon is termed "imitation" -- generation of images with recognizable similarity to training images. In this work we study the relationship between a concept's frequency and the ability of a model to imitate it. We seek to determine the point at which a model has observed enough training instances to imitate a concept -- the imitation threshold. We posit this question as a new problem: Finding the Imitation Threshold (FIT) and propose an efficient approach that estimates the imitation threshold without incurring the colossal cost of training multiple models from scratch. We experiment with two domains (human face and art style imitation), three datasets, three text-to-image models, and two pre-training datasets. Our results reveal that the imitation threshold of these models is in the range of 200-600 images, depending on the domain and the model. These imitation thresholds provide an empirical basis for copyright violation claims and act as a guiding principle for institutions training text-to-image models that aim to comply with copyright and privacy laws. We will release the code and data used upon publication.

Chat is not available.