Over the past two decades, there have been a growing number of image fusion solutions; however, there has not been a comprehensive book from which to teach standard image fusion methods. There are very few books that follow a textbook style that elaborates the entire process, from concepts and theory to evaluation and application. A textbook is especially useful to train beginners.
This book was written to provide readers with an understanding of image fusion techniques with basic principles, common examples, and known methods. Common examples are presented to interest any reader in the fundamentals. Although not all methods are extensively covered, the book aims to provide a students, practitioners, and researchers a background in proven techniques. Undergraduate training in engineering or science is recommended to appreciate concepts such as linear algebra and image processing.
The second motivation for the text was to organize the terminology, results, and techniques. The book and the associated software provide readers the opportunity to explore common image fusion methods, such as how to combine multiband images to enhance computer vision and human vision for applications such as face recognition and scene understanding.
The third motivation was to provide a baseline in performance evaluation of image fusion methods. Most publications concentrate on image fusion methods, although some quality metrics are used for comparison. Very few publications provide a comprehensive overview of fusion metrics and a comparison of objective metrics and subjective evaluations. Throughout this book, examples are shown and an array of metrics are presented that help establish the capabilities of image fusion. Different motivational applications might use some or none of the metrics, but the goal of the book is to start formalizing image fusion evaluation.
This book presents concepts, methods, evaluations, and applications of multispectral image fusion and night-vision colorization organized into four areas: (1) concepts, (2) theory, (3) evaluation, and (4) applications. Two primary multiscale fusion approaches - image pyramid and wavelet transform - are elaborated as applied to several examples, including face matching, biomedical imaging, and night vision. Using these examples, multiple-level fusion is demonstrated for pixel-, feature-, score-, and decision-level fusion. Image fusion comparisons are highlighted, including data, metrics, and analytics. Finally, the book also addresses a topic not highlighted elsewhere: techniques for evaluation, either objectively with computer metrics or subjectively by human users. An appendix includes online resources, including example data and code.
Chapter 1 describes the motivation of performing image fusion. An overview of fusion advantages is presented to endorse the idea of practical uses of image fusion.
Part I includes three chapters to present the background information and basic concepts of image fusion. Chapter 2 briefly surveys the field of image fusion. Chapter 3 discusses image fusion as it exists in biological vision, whereas Chapter 4 addresses certain sensor, object, and environmental operating conditions.
Part II describes image fusion theory in four chapters. Chapter 5 reviews image analysis techniques that form a processing pipeline for pattern-recognition tasks such as image fusion. Chapter 6 covers the information fusion approaches at different levels. Commonly used image fusion methods are described in Chapter 7. Chapter 8 is dedicated to a night-vision colorization technique that uses multispectral images.
Part III consists of two chapters dedicated to quantitative evaluation and qualitative evaluation. There are many publications inventing new image fusion methods. However, image fusion evaluation is needed to determine the comparative advantages of new methods. Quantitative metrics are described in Chapter 9 to objectively evaluate the quality of fused images including both grayscale and colorized imagers. Qualitative evaluation methods are discussed in Chapter 10 to conduct subjective assessments on fused imagery, which is crucial to military actions, medical applications, and colored imagery.
Part IV presents several fusion applications illustrated with off-focal, medical, terahertz, night-vision, and facial images. Chapter 11 concentrates on biometric applications, where a face-recognition example provides a complete illustration of multiple-level fusion. Chapter 12 includes an iterative wavelet-fusion example, a biomedical application of fusing magnetic resonance imaging scans, and terahertz and visual image fusion for concealed-object detection. Chapter 13 presents brief summaries of fusion methods, metrics, and examples.
The idea of writing a book was inspired by a SPIE short course - "Multispectral Image Fusion and Night Vision Colorization" - taught by Zheng and Blasch at the SPIE Defense and Commercial Sensing conference since 2014. Course attendees encouraged the writing of a textbook; furthermore, there was interest in a summary of night-vision-colorization techniques due to the growing needs of commercial operations. The comparison and evaluation of these techniques are unique features of this book.
This is the first comprehensive text for teaching image fusion, and we hope others improve on the techniques to make image fusion methods more common.
Yufeng Zheng
Erik Blasch
Zheng Liu
March 2018
