Skip to yearly menu bar Skip to main content


Poster
in
Workshop: NeurIPS 2023 Workshop on Machine Learning for Creativity and Design

Lasagna: Layered Score Distillation for Disentangled Image Editing

Dina Bashkirova · Arijit Ray · Rupayan Mallick · Sarah Bargal · Jianming Zhang · Ranjay Krishna · Kate Saenko


Abstract:

Recent text-guided image editing methods achieve great results on a variety of edit types, however, they fail to perform edits that are underrepresented in the training data, such as relighting. Methods that involve finetuning on paired supervised data often fail to preserve the input semantics on out-of-distribution examples, especially if the amount of training data is scarce. In this paper, we propose Lasagna, a method for disentangled image editing that distills the prior of a finetuned diffusion model in a separate visual layer. Lasagna uses score distillation to learn a plausible edit and preserves the semantics of the input by restricting the layer composition function. We show that Lasagna achieves superior shading quality compared to the state-of-the-art text-guided editing methods.

Chat is not available.