This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (201)  • Contact me when this article is cited  Related Content  • Similar articles in this journal  Social Bookmarking   What's this?

Findings From a National Sample

Craig A. Beam, PhD; Peter M. Layde, MD, MSc; Daniel C. Sullivan, MD

Arch Intern Med. 1996;156(2):209-213.

Abstract
Objective
To evaluate the effectiveness of screening mammography by estimating the variability in radiologists' ability to detect breast cancer within the US population of radiologists at mammography centers accredited by the American College of Radiology.

Methods
A two-way sample survey design was used as follows. Fifty mammography centers having an American College of Radiology—accredited unit were randomly sampled from across the United States. One hundred eight radiologists from these centers gave blinded interpretation to the same set of 79 randomly selected screening mammograms. The mammograms were from women who had been screened at a large screening center. Before their sampling, these women had been stratified by their breast disease status, established either by biopsy or by 2-year follow-up. Rates of biopsy recommendations were summarized by the mean, median, minimum, maximum, and range of sensitivity and specificity. Overall cancer detection ability was summarized by similar statistics for receiver operating characteristic curve areas. Ninety-five percent lower confidence bounds on the ranges in accuracy measures were established by bootstrapping.

Results
There is a range of at least 40% among US radiologists in their screening sensitivity. There is a range of at least 45% in the rates at which women without breast cancer are recommended for biopsy. As indicated by receiver operating characteristic curve areas, the ability of radiologists to detect cancer mammograms varies by as much as 11%.

Conclusions
Our findings indicate that there is wide variability in the accuracy of mammogram interpretation in the population of US radiologists. Current accreditation programs that certify the technical quality of radiographic equipment and images but not the accuracy of the interpretation given to mammograms may not be sufficient to help mammography fully realize its potential to reduce breast cancer mortality.

(Arch Intern Med. 1996;156:209-213)

Author Affiliations
From the Department of Family and Community Medicine, Medical College of Wisconsin, Milwaukee (Drs Beam and Layde), and the Department of Radiology, Duke University Medical Center, Durham, NC (Dr Sullivan). Dr Sullivan is now with the Department of Radiology, University of Pennsylvania Medical Center, Philadelphia.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Receiver Operating Characteristic Analysis for the Evaluation of Diagnosis and Prediction
Gatsonis
Radiology 2009;253:593-596.
FULL TEXT  

Variability in Interpretive Performance at Screening Mammography and Radiologists' Characteristics Associated with Accuracy
Elmore et al.
Radiology 2009;253:641-651.
ABSTRACT | FULL TEXT  

Quality Initiatives@;DELIM@;Continuing Medical Education: Developing a Radiology Quality and Safety Program: A Primer1
Johnson et al.
RadioGraphics 2009;29:951-959.
ABSTRACT | FULL TEXT  

Short-Term Hormone Therapy Suspension and Mammography Recall: A Randomized Trial
Buist et al.
ANN INTERN MED 2009;150:752-765.
ABSTRACT | FULL TEXT  

Developing a Radiology Quality and Safety Program: A Primer
Johnson et al.
RadioGraphics 2009;0:294095006.
ABSTRACT | FULL TEXT  

Comparison of Soft-copy and Hard-copy Reading for Full-Field Digital Mammography
Nishikawa et al.
Radiology 2009;251:41-49.
ABSTRACT | FULL TEXT  

Radiologists' Performance and Their Enjoyment of Interpreting Screening Mammograms
Geller et al.
Am. J. Roentgenol. 2009;192:361-369.
ABSTRACT | FULL TEXT  

The Natural History of Breast Cancer
Kaplan and Porzsolt
Arch Intern Med 2008;168:2302-2303.
FULL TEXT  

The "Laboratory" Effect: Comparing Radiologists' Performance and Variability during Prospective Clinical and Laboratory Mammography Interpretations
Gur et al.
Radiology 2008;249:47-53.
ABSTRACT | FULL TEXT  

Imaging as a Quantitative Science
Sullivan
Radiology 2008;248:328-332.
FULL TEXT  

References
JOURNAL OF THE ICRU 2008;8:57-62.
 

The Value of Bayes Theorem in the Interpretation of Subjective Diagnostic Findings: What Can We Learn from Agreement Studies?
Sadatsafavi et al.
Med Decis Making 2007;27:735-743.
ABSTRACT  

Errors in Cancer Diagnosis: Current Understanding and Future Directions
Singh et al.
JCO 2007;25:5009-5018.
ABSTRACT | FULL TEXT  

Effect of Slice Thickness and Primary 2D Versus 3D Virtual Dissection on Colorectal Lesion Detection at CT Colonography in 452 Asymptomatic Adults
Johnson et al.
Am. J. Roentgenol. 2007;189:672-680.
ABSTRACT | FULL TEXT  

Randomized Trial of Screen-Film versus Full-Field Digital Mammography with Soft-Copy Reading in Population-based Screening Program: Follow-up and Final Results of Oslo II Study
Skaane et al.
Radiology 2007;244:708-717.
ABSTRACT | FULL TEXT  

The More Eyes, the Better to See? From Double to Quadruple Reading of Screening Mammograms
Elmore and Brenner
JNCI J Natl Cancer Inst 2007;99:1141-1143.
FULL TEXT  

Breast Cancer Detection Rate: Designing Imaging Trials to Demonstrate Improvements
Jiang et al.
Radiology 2007;243:360-367.
ABSTRACT | FULL TEXT  

Accuracy of Diagnostic Procedures: Has It Improved Over the Past Five Decades?
Berlin
Am. J. Roentgenol. 2007;188:1173-1178.
FULL TEXT  

New Methodological Tools for Multiple-Reader ROC Studies
Obuchowski
Radiology 2007;243:10-12.
FULL TEXT  

Effect of Computer-Aided Detection on Independent Double Reading of Paired Screen-Film and Full-Field Digital Screening Mammograms
Skaane et al.
Am. J. Roentgenol. 2007;188:377-384.
ABSTRACT | FULL TEXT  

Performance Parameters for Screening and Diagnostic Mammography in a Community Practice: Are There Differences Between Specialists and General Radiologists?
Leung et al.
Am. J. Roentgenol. 2007;188:236-241.
ABSTRACT | FULL TEXT  

Testing the effect of computer-assisted detection on interpretive performance in screening mammography.
Taplin et al.
Am. J. Roentgenol. 2006;187:1475-1482.
ABSTRACT | FULL TEXT  

Performance Benchmarks for Screening Mammography
Rosenberg et al.
Radiology 2006;241:55-66.
ABSTRACT | FULL TEXT  

Computer-aided Detection in the United Kingdom National Breast Screening Programme: Prospective Study
Khoo et al.
Radiology 2005;237:444-449.
ABSTRACT | FULL TEXT  

Breast Lesion Detection and Classification: Comparison of Screen-Film Mammography and Full-Field Digital Mammography with Soft-copy Reading--Observer Performance Study
Skaane et al.
Radiology 2005;237:37-44.
ABSTRACT | FULL TEXT  

The Use of Batch Reading to Improve the Performance of Screening Mammography
Burnside et al.
Am. J. Roentgenol. 2005;185:790-796.
ABSTRACT | FULL TEXT  

Biomedical Imaging Research Opportunities Workshop II: Report and Recommendations
Partain et al.
Radiology 2005;236:389-403.
FULL TEXT  

Appealing Images: Magnetic Resonance Imaging and the Production of Authoritative Knowledge
Joyce
Social Studies of Science 2005;35:437-462.
ABSTRACT  

Performance Benchmarks for Diagnostic Mammography
Sickles et al.
Radiology 2005;235:775-790.
ABSTRACT | FULL TEXT  

Lessons from my dinners with the giants of modern image science
Wagner
Radiat Prot Dosimetry 2005;114:4-10.
ABSTRACT | FULL TEXT  

New Developments in Lung Cancer Screening
Mulshine
JCO 2005;23:3198-3202.
ABSTRACT | FULL TEXT  

Screening for Breast Cancer
Elmore et al.
JAMA 2005;293:1245-1256.
ABSTRACT | FULL TEXT  

Physician Predictors of Mammographic Accuracy
Smith-Bindman et al.
JNCI J Natl Cancer Inst 2005;97:358-367.
ABSTRACT | FULL TEXT  

ROC Analysis
Obuchowski
Am. J. Roentgenol. 2005;184:364-372.
FULL TEXT  

Accuracy of Screening Mammography Interpretation by Characteristics of Radiologists
Barlow et al.
JNCI J Natl Cancer Inst 2004;96:1840-1850.
ABSTRACT | FULL TEXT  

Reader Variability in Mammography and Its Implications for Expected Utility over the Population of Readers and Cases
Wagner et al.
Med Decis Making 2004;24:561-572.
ABSTRACT  

Computer-aided Detection Performance in Mammographic Examination of Masses: Assessment
Gur et al.
Radiology 2004;233:418-423.
ABSTRACT | FULL TEXT  

Radiologist Uncertainty and the Interpretation of Screening
Carney et al.
Med Decis Making 2004;24:255-264.
ABSTRACT  

How Many Observers Are Needed in Clinical Studies of Medical Imaging?
Obuchowski
Am. J. Roentgenol. 2004;182:867-869.
FULL TEXT  

On the benefits and harms of screening for breast cancer
Gotzsche
Int J Epidemiol 2004;33:56-64.
FULL TEXT  

Population-based Mammography Screening: Comparison of Screen-Film and Full-Field Digital Mammography with Soft-Copy Reading--Oslo I Study
Skaane et al.
Radiology 2003;229:877-884.
ABSTRACT | FULL TEXT  

Special Topics III: Bias
Obuchowski
Radiology 2003;229:617-621.
ABSTRACT | FULL TEXT  

Evaluation of Proscriptive Health Care Policy Implementation in Screening Mammography
Beam et al.
Radiology 2003;229:534-540.
ABSTRACT | FULL TEXT  

International Variation in Screening Mammography Interpretations in Community-Based Programs
Elmore et al.
JNCI J Natl Cancer Inst 2003;95:1384-1393.
ABSTRACT | FULL TEXT  

Generalists versus Specialists in Mammography [letter] * Dr Sickles and colleagues respond:
Guenin et al.
Radiology 2003;227:609-611.
FULL TEXT  

The Central Role of Receiver Operating Characteristic (ROC) Curves in Evaluating Tests for the Early Detection of Cancer
Baker
JNCI J Natl Cancer Inst 2003;95:511-515.
FULL TEXT  

Association of Volume and Volume-Independent Factors With Accuracy in Screening Mammogram Interpretation
Beam et al.
JNCI J Natl Cancer Inst 2003;95:282-290.
ABSTRACT | FULL TEXT  

Diagnostic Accuracy of Digital Mammography in Patients with Dense Breasts Who Underwent Problem-solving Mammography: Effects of Image Processing and Lesion Type
Cole et al.
Radiology 2003;226:153-160.
ABSTRACT | FULL TEXT  

Lung cancer * 2: Screening and early diagnosis of lung cancer
Mulshine and Smith
Thorax 2002;57:1071-1078.
ABSTRACT | FULL TEXT  

Screening for Preclinical Disease: Test and Disease Characteristics
Herman et al.
Am. J. Roentgenol. 2002;179:825-831.
FULL TEXT  

Screening Mammograms by Community Radiologists: Variability in False-Positive Rates
Elmore et al.
JNCI J Natl Cancer Inst 2002;94:1373-1380.
ABSTRACT | FULL TEXT  

Performance Parameters for Screening and Diagnostic Mammography: Specialist and General Radiologists
Sickles et al.
Radiology 2002;224:861-869.
ABSTRACT | FULL TEXT  

Does Training in the Breast Imaging Reporting and Data System (BI-RADS) Improve Biopsy Recommendations or Feature Analysis Agreement with Experienced Breast Imagers at Mammography?
Berg et al.
Radiology 2002;224:871-880.
ABSTRACT | FULL TEXT  

Performance of Diagnostic Mammography for Women With Signs or Symptoms of Breast Cancer
Barlow et al.
JNCI J Natl Cancer Inst 2002;94:1151-1159.
ABSTRACT | FULL TEXT  

Determining Sensitivity of Mammography from Screening Data, Cancer Incidence, and Receiver-Operating Characteristic Curve Parameters
Boone et al.
Med Decis Making 2002;22:228-237.
ABSTRACT  

The Fundamental Issue of Diagnosis
Farmer
Arch Dermatol 2002;138:684-685.
FULL TEXT  

Improving the Accuracy of Mammography: Volume and Outcome Relationships
Esserman et al.
JNCI J Natl Cancer Inst 2002;94:369-375.
ABSTRACT | FULL TEXT  

Potential of Computer-aided Diagnosis to Reduce Variability in Radiologists' Interpretations of Mammograms Depicting Microcalcifications
Jiang et al.
Radiology 2001;220:787-794.
ABSTRACT | FULL TEXT  

Providing Professional Mammography Services: Financial Analysis
Enzmann et al.
Radiology 2001;219:467-473.
ABSTRACT | FULL TEXT  

Rates and Causes of Disagreement in Interpretation of Full-Field Digital Mammography and Film-Screen Mammography in a Diagnostic Setting
Venta et al.
Am. J. Roentgenol. 2001;176:1241-1248.
ABSTRACT | FULL TEXT  

Mammography in 53,803 Women from the New Hampshire Mammography Network
Poplack et al.
Radiology 2000;217:832-840.
ABSTRACT | FULL TEXT  

Statistical description of interrater variability in ordinal ratings
Nelson and Pepe
Stat Methods Med Res 2000;9:475-496.
ABSTRACT  

Power Doppler Imaging: Initial Evaluation as a Screening Examination for Carotid Artery Stenosis
Bluth et al.
Radiology 2000;215:791-800.
ABSTRACT | FULL TEXT  

Breast Imaging Reporting and Data System: Inter- and Intraobserver Variability in Feature Analysis and Final Assessment
Berg et al.
Am. J. Roentgenol. 2000;174:1769-1777.
ABSTRACT | FULL TEXT  

Accuracy of Screening Mammography Using Single Versus Independent Double Interpretation
Taplin et al.
Am. J. Roentgenol. 2000;174:1257-1262.
ABSTRACT | FULL TEXT  

Current Status of Full-Field Digital Mammography1
Pisano
Radiology 2000;214:26-28.
FULL TEXT  

Dynamic Breast MR Imaging: Are Signal Intensity Time Course Data Useful for Differential Diagnosis of Enhancing Lesions?
Kuhl et al.
Radiology 1999;211:101-110.
ABSTRACT | FULL TEXT  

Avoiding the Unintended Consequences of Growth in Medical Care: How Might More Be Worse?
Fisher and Welch
JAMA 1999;281:446-453.
ABSTRACT | FULL TEXT  

Analysis of clustered data in receiver operating characteristic studies
Beam
Stat Methods Med Res 1998;7:324-336.
ABSTRACT  

Recommendations for Follow-up Care of Individuals With an Inherited Predisposition to Cancer: II. BRCA1 and BRCA2
Burke et al.
JAMA 1997;277:997-1003.
ABSTRACT  

WIDE VARIATION IN RADIOLOGISTS' INTERPRETATION OF MAMMOGRAMS
JWatch General 1996;1996:2-2.
FULL TEXT