Comparison of two stand-alone CADe systems at multiple operating points

Berkman Sahiner; Weijie Chen; Aria Pezeshk; Nicholas Petrick

doi:10.1117/12.2082123

17 March 2015 Comparison of two stand-alone CADe systems at multiple operating points

Berkman Sahiner, Weijie Chen, Aria Pezeshk, Nicholas Petrick

Proceedings Volume 9416, Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment; 941612 (2015) https://doi.org/10.1117/12.2082123
Event: SPIE Medical Imaging, 2015, Orlando, Florida, United States

Abstract

Computer-aided detection (CADe) systems are typically designed to work at a given operating point: The device displays a mark if and only if the level of suspiciousness of a region of interest is above a fixed threshold. To compare the standalone performances of two systems, one approach is to select the parameters of the systems to yield a target false-positive rate that defines the operating point, and to compare the sensitivities at that operating point. Increasingly, CADe developers offer multiple operating points, which necessitates the comparison of two CADe systems involving multiple comparisons. To control the Type I error, multiple-comparison correction is needed for keeping the family-wise error rate (FWER) less than a given alpha-level. The sensitivities of a single modality at different operating points are correlated. In addition, the sensitivities of the two modalities at the same or different operating points are also likely to be correlated. It has been shown in the literature that when test statistics are correlated, well-known methods for controlling the FWER are conservative. In this study, we compared the FWER and power of three methods, namely the Bonferroni, step-up, and adjusted step-up methods in comparing the sensitivities of two CADe systems at multiple operating points, where the adjusted step-up method uses the estimated correlations. Our results indicate that the adjusted step-up method has a substantial advantage over other the two methods both in terms of the FWER and power.

Citation Download Citation

Berkman Sahiner, Weijie Chen, Aria Pezeshk, and Nicholas Petrick "Comparison of two stand-alone CADe systems at multiple operating points", Proc. SPIE 9416, Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment, 941612 (17 March 2015); https://doi.org/10.1117/12.2082123

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Computer aided diagnosis and therapy

Selenium

Computer aided design

Error analysis

CAD systems

Statistical analysis

Binary data

Show All Keywords

Keywords/Phrases

Search In:

Publication Years