Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Medical Imaging meets NeurIPS

LC-SD: Realistic Endoscopic Image Generation with Stable Diffusion and ControlNet

Joanna Kaleta · Diego Dall'Alba · Szymon PÅ‚otka · Przemyslaw Korzeniowski


Abstract:

Computer-assisted surgical systems provide support information to the surgeon, which can improve the execution and overall outcome of the procedure. These systems are based on deep learning models that are trained on complex and challenging-to-annotate data. Generating synthetic data can overcome these limitations, but it is necessary to reduce the domain gap between real and synthetic data. We propose a method for image-to-image translation based on a Stable Diffusion model, which generates realistic images from synthetic data. Compared to previous works, the proposed method is better suited for clinical application, requiring a much smaller amount of input data and allowing finer control over the generation of details by introducing different variants of supporting control networks. The proposed method is applied in the context of laparoscopic cholecystectomy, using synthetic and real public data. It achieves a mean Intersection over Union of 69.76\%, significantly improving the baseline results (69.76 vs. 42.21\%). The proposed method for translating synthetic images into images with realistic characteristics will enable the training of deep learning methods that can generalize optimally to real-world contexts, thereby improving computer-assisted intervention guidance systems.

Chat is not available.