An infrared and visible image fusion method based on multi-head attention feature and autoencoder

Chen-wei Yang; Xian-zhi Chen; Yuan Zhao

doi:10.1117/12.2690223

17 August 2023 An infrared and visible image fusion method based on multi-head attention feature and autoencoder

Chen-wei Yang, Xian-zhi Chen, Yuan Zhao

Proceedings Volume 12757, 3rd International Conference on Laser, Optics, and Optoelectronic Technology (LOPET 2023); 1275737 (2023) https://doi.org/10.1117/12.2690223
Event: 3rd International Conference on Laser, Optics and Optoelectronic Technology (LOPET 2023), 2023, Kunming, China

Abstract

In recent years, researchers have proposed many image fusion methods based on convolutional autoencoder structure, which can make the feature map extracted by convolutional neural network more suitable for image fusion task through specific network structure, specially designed functional modules and loss functions, and achieve satisfactory results. However, these methods all use feature maps of different scales for direct fusion, and usually the information between different parts of an image is related. If the feature map is directly fused, only the local features of the image are used, while the global information of the image is ignored. In order to improve this defect, this paper will first introduce the Transformer model with multi-head self-attention as the core, how Transformer introduced into the image processing task, combined with the previous autoencoder fusion framework, propose an image fusion model based on attention feature, and design and conduct comparative experiments. Experimental results show that this method can effectively obtain the global features of the image, so as to further improve the quality of infrared and visible image fusion.

Citation Download Citation

Chen-wei Yang, Xian-zhi Chen, and Yuan Zhao "An infrared and visible image fusion method based on multi-head attention feature and autoencoder", Proc. SPIE 12757, 3rd International Conference on Laser, Optics, and Optoelectronic Technology (LOPET 2023), 1275737 (17 August 2023); https://doi.org/10.1117/12.2690223

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available