Components of variance in ROC analysis of CADx classifier performance: II. Applications of the bootstrap

Robert F. Wagner; Heang-Ping Chan; Berkman Sahiner; Nicholas Petrick; Joseph T. Mossoba

doi:10.1117/12.348608

21 May 1999 Components of variance in ROC analysis of CADx classifier performance: II. Applications of the bootstrap

Robert F. Wagner, Heang-Ping Chan, Berkman Sahiner, Nicholas Petrick, Joseph T. Mossoba

Proceedings Volume 3661, Medical Imaging 1999: Image Processing; (1999) https://doi.org/10.1117/12.348608
Event: Medical Imaging '99, 1999, San Diego, CA, United States

Abstract

We review components-of-variance models for the uncertainty in estimates of the area under the ROC curve, A_z, for the case of classical discriminants where we wish the uncertainty to generalize to a population of training cases as well as to a population of testing cases. A key observation from our previous work facilitates the use of resampling strategies to analyze a finite data set and classifier in terms of the components-of-variance models. In particular, we demonstrate the use of the statistical bootstrap in combination with a four-term variance model to solve for the contributions of the uncertainty in A_z that result from a given finite training sample, a given finite test sample, and their interaction. At the same time one obtains an expression from which one can predict the change in uncertainty in estimates of A_z that would result from a given change in the number of training samples and change in the number of test samples. This expression provides a quantitative design tool for estimating the size that would be required in a larger pivotal study from the results of a smaller pilot study for the purpose of achieving a desired precision in A_z and the desired generalizability.

Citation Download Citation

Robert F. Wagner, Heang-Ping Chan, Berkman Sahiner, Nicholas Petrick, and Joseph T. Mossoba "Components of variance in ROC analysis of CADx classifier performance: II. Applications of the bootstrap", Proc. SPIE 3661, Medical Imaging 1999: Image Processing, (21 May 1999); https://doi.org/10.1117/12.348608

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available