Poster
in
Workshop: NeurIPS 2023 Workshop on Machine Learning for Creativity and Design
Lasagna: Layered Score Distillation for Disentangled Image Editing
Dina Bashkirova · Arijit Ray · Rupayan Mallick · Sarah Bargal · Jianming Zhang · Ranjay Krishna · Kate Saenko
Recent text-guided image editing methods achieve great results on a variety of edit types, however, they fail to perform edits that are underrepresented in the training data, such as relighting. Methods that involve finetuning on paired supervised data often fail to preserve the input semantics on out-of-distribution examples, especially if the amount of training data is scarce. In this paper, we propose Lasagna, a method for disentangled image editing that distills the prior of a finetuned diffusion model in a separate visual layer. Lasagna uses score distillation to learn a plausible edit and preserves the semantics of the input by restricting the layer composition function. We show that Lasagna achieves superior shading quality compared to the state-of-the-art text-guided editing methods.