Analyzing the role of visual structure in the recognition of natural image content with multi-scale SSIM

David M. Rouse; Sheila S. Hemami

doi:10.1117/12.768060

14 February 2008 Analyzing the role of visual structure in the recognition of natural image content with multi-scale SSIM

David M. Rouse, Sheila S. Hemami

Author Affiliations +

Proceedings Volume 6806, Human Vision and Electronic Imaging XIII; 680615 (2008) https://doi.org/10.1117/12.768060
Event: Electronic Imaging, 2008, San Jose, California, United States

Abstract

Natural images are meaningful to humans - the physical world exhibits statistical regularities that permit the human visual system (HVS) to infer useful interpretations. These regularities communicate the visual structure of the physical world and govern the statistics of images (image structure). A signal processing framework is sought to analyze image characteristics for a relationship with human interpretation. This work investigates the first step toward an objective visual information evaluation: predicting the recognition threshold of different image representations. Given a image sequence, whose images begin as unrecognizable and are gradually refined to include more information according to some measure, the recognition threshold corresponds to first the image in the sequence in which an observer accurately identifies the content. Sequences are produced using two types of image representations: signal-based and visual structure preserving. Signal-based representations add information as dictated by conventional mathematical characterizations of images based on models of low-level HVS processing and use basis functions as the basic image components. Visual structure preserving representations add information to images attributed to visual structure and attempt to mimic higher-level HVS processing by considering the scene's objects as the basic image components. An experiment is conducted to identify the recognition threshold image. Several full-reference perceptual quality assessment algorithms are evaluated in terms of their ability to predict the recognition threshold of different image representations. The cross-correlation component of a modified version of the multi-scale structural similarity (MS-SSIM) metric, denoted MS-SSIM*, exhibits a better overall correlation with the signal-based and visual structure preserving representations' average recognition thresholds than the standard MS-SSIM cross-correlation component. These findings underscore the significance of visual structure in recognition and advocate a multi-scale image structure analysis for a rudimentary evaluation of visual information.

Citation Download Citation

David M. Rouse and Sheila S. Hemami "Analyzing the role of visual structure in the recognition of natural image content with multi-scale SSIM", Proc. SPIE 6806, Human Vision and Electronic Imaging XIII, 680615 (14 February 2008); https://doi.org/10.1117/12.768060

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available