Interpreting the ROC curve from the radiologist's point of view: The "Diagnostician Operating Choice" (DOC) curve

C. A. Beam, E. F. Conant, E. A. Krupinski, H. L. Kundel, E. A. Sickles

Research output: Contribution to journalConference articlepeer-review


We introduce an interesting interpretation of the ROC Curve that, subsequently, opens a new research paradigm. We define the "Diagnostician Operating Choice" (DOC) Curve to be the set of all (True Positive Probability/True Negative Probability) or ("skill in diseased population"/"skill in non-diseased population" when considered from the diagnostician's perspective) options made available to a particular radiologist when interpreting a particular diagnostic technology. The DOC Curve is, thus, the choice set presented to the diagnostician by their interaction with the technology. This new paradigm calls for tools that can measure the particular choice set of any particular individual radiologist interpreting a particular technology when applied in a particular clinical setting. Fundamental requirements for this paradigm are for the DOC Curve to be unique to individuals and constant across similar experimental conditions. To investigate constancy, we analyzed data from a reading study of 10 radiologists. Each radiologist interpreted the same set of 148 screening mammograms twice using a modified version of BI-RADS. ROC Curves for each radiologist were computed and compared between the two reading occasions with the CORROC2 program. None of the areas were statistically significantly different (p<0.05), providing confirmation (but not proof) of constancy across the two reading conditions. The DOC Curve paradigm suggests new areas of research focusing on the behavior in individuals interacting with technology. A clear need is for more efficient estimation of individual DOC Curves based on limited case sets. Paradoxically, the answer to this last problem might lie in using large population-based ("MRMC") studies to develop highly efficient and externally validated standardized testing tools for assessment of the individual.

Original languageEnglish (US)
Article number02
Pages (from-to)8-14
Number of pages7
JournalProgress in Biomedical Optics and Imaging - Proceedings of SPIE
StatePublished - 2005
EventMedical Imaging 2005 - Image Perception, Observer Performance, and Technology Assessment - San Diego, CA, United States
Duration: Feb 15 2005Feb 17 2005


  • Diagnosis
  • Observer Variability
  • ROC Curve

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Interpreting the ROC curve from the radiologist's point of view: The "Diagnostician Operating Choice" (DOC) curve'. Together they form a unique fingerprint.

Cite this