Lightweight multi-branch feature fusion network for multimodal medical image fusion

Yu Shao; Lei Yu

doi:10.1117/12.3049577

8 November 2024 Lightweight multi-branch feature fusion network for multimodal medical image fusion

Yu Shao, Lei Yu

Proceedings Volume 13416, Fourth International Conference on Advanced Algorithms and Neural Networks (AANN 2024); 1341626 (2024) https://doi.org/10.1117/12.3049577
Event: 2024 4th International Conference on Advanced Algorithms and Neural Networks, 2024, Qingdao, China

Abstract

Most existing medical image fusion methods, which are based on deep learning achieve satisfactory results by using complex network architectures and stacking numerous modules. However, these methods often overlook the application scenarios of multimodal medical image fusion. The complex model structure and large parameter load make deployment on mobile devices extremely challenging. Moreover, it is unreasonable to consume substantial computational resources at the low-level image processing stage if the method is to be applied to downstream computational tasks. We have innovatively designed a lightweight multi-branch feature fusion network for multimodal medical image fusion. This method has a lower parameter count and extremely fast forward inference speed. This is due to our designed multi-branch feature channel segmentation method, which divide-and-conquer feature extraction with different receptive fields. We also use a channel attention mechanism based on contrast awareness to fuse and reduce the dimensionality of the feature maps, preserving ample source image information while reducing computational load. Finally, the fusion image reconstruction is completed through a sliding window attention mechanism combined with long-range feature dependencies.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yu Shao and Lei Yu "Lightweight multi-branch feature fusion network for multimodal medical image fusion", Proc. SPIE 13416, Fourth International Conference on Advanced Algorithms and Neural Networks (AANN 2024), 1341626 (8 November 2024); https://doi.org/10.1117/12.3049577

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available