LSGDM: shadow generation for composite image with latent diffusion model

Yong Lu; Guangxi Chen; Xiaolin Xiang

doi:10.1117/12.3038549

23 August 2024 LSGDM: shadow generation for composite image with latent diffusion model

Yong Lu, Guangxi Chen, Xiaolin Xiang

Proceedings Volume 13250, Fourth International Conference on Image Processing and Intelligent Control (IPIC 2024); 132500R (2024) https://doi.org/10.1117/12.3038549
Event: 4th International Conference on Image Processing and Intelligent Control (IPIC 2024), 2024, Kuala Lumpur, Malaysia

Abstract

Composite images generated through simply merging objects with background images often demonstrate a pronounced lack of realism due to the lack of shadows, an important visual element. Therefore, shadow generation plays a critical role within the domain of image synthesis. Existing shadow generation methods primarily employ Generative Adversarial Networks (GANs) as the underlying framework. However, GANs often overlook critical guiding features in background images due to mode collapse, leading to poor shadow generation quality. To address this concern, we have designed a two-stage shadow generation method based on Latent Diffusion Model (LDM). The non-adversarial training process allows the diffusion model to effectively avoid mode collapse. Specifically, in the first stage, we introduce a non-local attention mechanism and design a focal loss function based on Gaussian activation to generate high-quality shadow masks. In the second stage, we treat the shadow shading task as a special style transfer task to extract shadow shading guide features from background images, and design a partitioned dynamic weight loss to shade the masked areas as shadows. We demonstrate the effectiveness of our model through a series of experiments on the DESOBA dataset.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yong Lu, Guangxi Chen, and Xiaolin Xiang "LSGDM: shadow generation for composite image with latent diffusion model", Proc. SPIE 13250, Fourth International Conference on Image Processing and Intelligent Control (IPIC 2024), 132500R (23 August 2024); https://doi.org/10.1117/12.3038549

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Shadows

Diffusion

RGB color model

Performance modeling

Image compression

Light sources and illumination

Data modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years