Effect of observer inattention in a detection task on ROC analysis

Yulei Jiang; William Sacks; Charles E. Metz

doi:10.1117/12.597597

6 April 2005 Effect of observer inattention in a detection task on ROC analysis

Yulei Jiang, William Sacks, Charles E. Metz

Proceedings Volume 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment; (2005) https://doi.org/10.1117/12.597597
Event: Medical Imaging, 2005, San Diego, California, United States

Abstract

ROC experiments generally assume observers to be fully attentive to the task of the experiment and, therefore, that the experiment measures observers' innate ability in performing the task. In a detection task, inattention can cause an observer to overlook signals or signal-like potential false positives in an image. In an ROC experiment that involves a detection task, inattention can either cause an observer to report a strong confidence for signal absent, or cause the observer to report on some other finding less salient than the finding that the observer fails to notice. The purpose of this study was to determine the effect of observer inattention on empirical ROC estimates and the appropriateness of the conventional binormal model for fitting such ROC curves. An experiment was designed in which observers were asked to detect a signal of a simple geometric shape and varying contrast in a background of Gaussian noise. The images sometimes also contained other objects of different geometric shapes in locations that did not overlap the signal. Observer inattention was simulated by blocking a quarter of each image from observer view. Results showed that observer inattention caused the empirical ROC curve to decrease, but curve fitting with the binormal model appeared to be equally accurate with and without observer inattention. However, a "proper" ROC model might be more appropriate than the binormal model because the binormal model invariably produced a "hook" near the upper-right corner of the ROC unit square that was not indicated by the empirical operating points. We conclude that observer inattention in a detection task can be evaluated as part of a conventional ROC experiment.

Citation Download Citation

Yulei Jiang, William Sacks, and Charles E. Metz "Effect of observer inattention in a detection task on ROC analysis", Proc. SPIE 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment, (6 April 2005); https://doi.org/10.1117/12.597597

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
4 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Signal detection

Data modeling

Mammography

Medical imaging

Analytical research

Cancer

Computer aided diagnosis and therapy

Show All Keywords

Keywords/Phrases

Search In:

Publication Years