0

The full content of Annals is available to subscribers

Subscribe/Learn More  >
Original Research |

Variability in Pathologists' Interpretations of Individual Breast Biopsy Slides: A Population PerspectiveVariability in Interpretations of Individual Breast Biopsy Slides

Joann G. Elmore, MD, MPH; Heidi D. Nelson, MD, MPH; Margaret S. Pepe, PhD; Gary M. Longton, MS; Anna N.A. Tosteson, ScD; Berta Geller, EdD; Tracy Onega, PhD; Patricia A. Carney, PhD; Sara L. Jackson, MD, MPH; Kimberly H. Allison, MD; and Donald L. Weaver, MD
[+] Article, Author, and Disclosure Information

This article was published at www.annals.org on 22 March 2016.


From University of Washington School of Medicine and Fred Hutchinson Cancer Research Center, Seattle, Washington; Providence Cancer Center, Providence Health & Services Oregon, and Oregon Health & Science University, Portland, Oregon; Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire; University of Vermont, Burlington, Vermont; and Stanford University School of Medicine, Stanford, California.

Note: Drs. Elmore and Pepe had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Disclaimer: The content of this article is solely the responsibility of the authors and does not necessarily represent the views of the National Cancer Institute or the National Institutes of Health.

Acknowledgment: The authors appreciate the efforts of the pathologists who participated in this study.

Financial Support: This work was supported by the National Cancer Institute of the National Institutes of Health (award numbers R01 CA140560, U01CA86082, U01 CA70013, and R01 CA172343) and by the National Cancer Institute-funded BCSC (award number HHSN261201100031C). The collection of cancer and vital status data used in this study was supported in part by several state public health departments and cancer registries throughout the United States. A full description of sources is available at www.breastscreening.cancer.gov/work/acknowledgement.html.

Disclosures: Dr. Elmore reports grants from the National Cancer Institute during the conduct of the study. Mr. Longton reports grants from the National Cancer Institute during the conduct of the study. Dr. Onega reports grants from the National Cancer Institute during the conduct of the study. Dr. Weaver reports grants from the National Cancer Institute during the conduct of the study. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOf InterestForms.do?msNum=M15-0964.

Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that she has no financial relationships or interests to disclose. Darren B. Taichman, MD, PhD, Executive Deputy Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Deborah Cotton, MD, MPH, Deputy Editor, reports that she has no financial relationships or interest to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Catharine B. Stack, PhD, MS, Deputy Editor for Statistics, reports that she has stock holdings in Pfizer.

Reproducible Research Statement:Study protocol, statistical code, and data set: Readers may contact the authors directly to discuss the study protocol, the statistical code, or the data set from which the results were derived (e-mail, jelmore@u.washington.edu).

Requests for Single Reprints: Joann G. Elmore, MD, MPH, University of Washington, Mailbox 359780, 325 Ninth Avenue, Seattle, WA 98104.

Current Author Addresses: Drs. Elmore and Jackson: University of Washington, Mailbox 359780, 325 Ninth Avenue, Seattle, WA 98104.

Dr. Nelson: Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Mail Code BICC, Portland, OR 97239.

Dr. Pepe and Mr. Longton: Program in Biostatistics and Biomathematics, Fred Hutchinson Cancer Research Center, M2-B500, 1100 Fairview Avenue North, PO Box 19024, Seattle, WA 98109-1024.

Dr. Tosteson: Geisel School of Medicine at Dartmouth, One Medical Center Drive (HB7505), Lebanon, NH 03756.

Dr. Geller: Family Medicine, University of Vermont, 1 South Prospect Street, Burlington, VT 05401.

Dr. Onega: Section of Biostatistics & Epidemiology, Geisel School of Medicine at Dartmouth, One Medical Center Drive (HB7937), Lebanon, NH 03756.

Dr. Carney: Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Mail Code FM, Portland, OR 97239.

Dr. Allison: Department of Pathology, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA 93195.

Dr. Weaver: Department of Pathology, University of Vermont, Courtyard at Given, 89 Beaumont Avenue, Burlington, VT 05405-0068.

Author Contributions: Conception and design: J.G. Elmore, H.D. Nelson, M.S. Pepe, A.N.A. Tosteson, B. Geller, P.A. Carney, K.H. Allison, D.L. Weaver.

Analysis and interpretation of the data: J.G. Elmore, H.D. Nelson, M.S. Pepe, G.M. Longton, A.N.A. Tosteson, B. Geller, T. Onega, P.A. Carney, S.L. Jackson, K.H. Allison, D.L. Weaver.

Drafting of the article: J.G. Elmore, H.D. Nelson, M.S. Pepe, A.N.A. Tosteson, B. Geller, T. Onega, P.A. Carney, S.L. Jackson, K.H. Allison, D.L. Weaver.

Critical revision of the article for important intellectual content: J.G. Elmore, H.D. Nelson, M.S. Pepe, G.M. Longton, A.N.A. Tosteson, B. Geller, T. Onega, P.A. Carney, K.H. Allison, D.L. Weaver.

Final approval of the article: J.G. Elmore, H.D. Nelson, M.S. Pepe, G.M. Longton, A.N.A. Tosteson, B. Geller, T. Onega, P.A. Carney, S.L. Jackson, K.H. Allison, D.L. Weaver.

Provision of study materials or patients: J.G. Elmore, T. Onega, P.A. Carney, D.L. Weaver.

Statistical expertise: M.S. Pepe, G.M. Longton, A.N.A. Tosteson.

Obtaining of funding: J.G. Elmore, M.S. Pepe, A.N.A. Tosteson, T. Onega, P.A. Carney, D.L. Weaver.

Administrative, technical, or logistic support: J.G. Elmore, S.L. Jackson, D.L. Weaver.

Collection and assembly of data: J.G. Elmore, H.D. Nelson, B. Geller, T. Onega, P.A. Carney, S.L. Jackson, D.L. Weaver.


Ann Intern Med. 2016;164(10):649-655. doi:10.7326/M15-0964
© 2016 American College of Physicians
Text Size: A A A

Background: The effect of physician diagnostic variability on accuracy at a population level depends on the prevalence of diagnoses.

Objective: To estimate how diagnostic variability affects accuracy from the perspective of a U.S. woman aged 50 to 59 years having a breast biopsy.

Design: Applied probability using Bayes' theorem.

Setting: B-Path (Breast Pathology) Study comparing pathologists' interpretations of a single biopsy slide versus a reference consensus interpretation from 3 experts.

Participants: 115 practicing pathologists (6900 total interpretations from 240 distinct cases).

Measurements: A single representative slide from each of the 240 cases was used to estimate the proportion of biopsies with a diagnosis that would be verified if the same slide were interpreted by a reference group of 3 expert pathologists. Probabilities of confirmation (predictive values) were estimated using B-Path Study results and prevalence of biopsy diagnoses for women aged 50 to 59 years in the Breast Cancer Surveillance Consortium.

Results: Overall, if 1 representative slide were used per case, 92.3% (95% CI, 91.4% to 93.1%) of breast biopsy diagnoses would be verified by reference consensus diagnoses, with 4.6% (CI, 3.9% to 5.3%) overinterpreted and 3.2% (CI, 2.7% to 3.6%) underinterpreted. Verification of invasive breast cancer and benign without atypia diagnoses is highly probable; estimated predictive values were 97.7% (CI, 96.5% to 98.7%) and 97.1% (CI, 96.7% to 97.4%), respectively. Verification is less probable for atypia (53.6% overinterpreted and 8.6% underinterpreted) and ductal carcinoma in situ (DCIS) (18.5% overinterpreted and 11.8% underinterpreted).

Limitations: Estimates are based on a testing situation with 1 slide used per case and without access to second opinions. Population-adjusted estimates may differ for women from other age groups, unscreened women, or women in different practice settings.

Conclusion: This analysis, based on interpretation of a single breast biopsy slide per case, predicts a low likelihood that a diagnosis of atypia or DCIS would be verified by a reference consensus diagnosis. This diagnostic grey zone should be considered in clinical management decisions in patients with these diagnoses.

Primary Funding Source: National Cancer Institute.

Figures

Grahic Jump Location
Figure 1.

Prevalence of breast biopsy diagnostic interpretations in the B-Path Study test set and among women aged 50 to 59 y having screening mammography in the United States.

Estimates of population prevalence are from the Breast Cancer Surveillance Consortium (6). B-Path = Breast Pathology; DCIS = ductal carcinoma in situ.

Grahic Jump Location
Grahic Jump Location
Figure 2.

Predicted proportions of breast biopsy interpretations (based on a single slide per case) that would be verified by the reference consensus panel interpretation or would be classified as overinterpretations or underinterpretations.

DCIS = ductal carcinoma in situ.

Grahic Jump Location
Grahic Jump Location
Figure 3.

Predicted outcomes per 100 breast biopsies, overall and by diagnostic category.

Each 100-slide grid demonstrates the predicted number of cases overinterpreted or underinterpreted relative to the reference consensus panel diagnosis, or verified. DCIS = ductal carcinoma in situ.

* Interpretation is more severe than the consensus reference panel.

† Interpretation is less severe than the consensus reference panel.

Grahic Jump Location

Tables

References

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment/Letter
Comment
Posted on June 15, 2016
William L. Wolfson, MD, FACP
Bishop, CA
Conflict of Interest: None Declared
Having practiced surgical pathology for 40 years I was fascinated to read the article by Elmore et. al. in the 17 May issue (Variability in Pathologists’ Interpretations of Individual Breast Biopsy Slides: A Population Perspective).
It is hardly news that the diagnosis of any breast lesion other than “totally benign” or “frankly invasive carcinoma” is fraught with irreproducibility. Furthermore, such irreproducibility extends beyond community practitioners to the realm of “expert breast pathologist”. Decades ago Juan Rosai did a study of five expert breast pathologists reviewing slides of 10 borderline lesions. On not one did all five agree.
Pathologists are well aware of this problem and over the years terms other than DCIS have been proposed – Mammary Intraepithelial Neoplasia, Ductal Intraepithelial Neoplasia ad infinitum. And many agree than such lesions should not include “carcinoma” in the diagnostic term (Dr. Sahla Mahsood has written extensively about this in both journals and newspapers). Interestingly, some patient advocacy groups object to this, believing it trivializes their condition.
This particular study suffers from a few obvious faults – no pathologist would ever think of signing out any sort of breast lesion on the basis of a single slide. There is no mention of the degree of agreement among the panel of three experts. As to therapeutic decision making pathologists do not claim to be a “gold standard” however much clinicians would like to have one. Most path reports on borderline lesions include so many caveats and disclaimers that second (third and fourth) opinions are commonplace. Attend a breast tumor board and you will hear a wide range of treatment and further diagnostic options discussed.
In short, the reproducibility for benign and malignant lesions is excellent. The reproducibility for less well defined lesions is in accord with their nature. Sad perhaps, but unavoidable at present even with the most elegant technologies we can bring to bear.

Sincerely,



William L. Wolfson, MD
FACP, FCAP, FASCP, FIAC
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)