Intended for healthcare professionals

  1. Research
  2. Evaluation of 12...
  3. Evaluation of 12 strategies for obtaining second opinions to improve interpretation of breast histopathology: simulation study
CCBYNC Open access
Research

Evaluation of 12 strategies for obtaining second opinions to improve interpretation of breast histopathology: simulation study

BMJ 2016; 353 doi: https://doi.org/10.1136/bmj.i3069 (Published 22 June 2016) Cite this as: BMJ 2016;353:i3069
  1. Joann G Elmore, professor1,
  2. Anna NA Tosteson, professor2 3,
  3. Margaret S Pepe, full member4,
  4. Gary M Longton, senior statistical analyst5,
  5. Heidi D Nelson, professor6,
  6. Berta Geller, research professor emerita7,
  7. Patricia A Carney, professor8,
  8. Tracy Onega, associate professor9,
  9. Kimberly H Allison, associate professor10,
  10. Sara L Jackson, clinical assistant professor11,
  11. Donald L Weaver, professor12
  1. 1Department of Medicine, University of Washington School of Medicine, Seattle, WA, USA
  2. 2The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Norris Cotton Cancer Center, Lebanon, NH, USA
  3. 3Department of Medicine, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
  4. 4Fred Hutchinson Cancer Research Center, Seattle, WA, USA
  5. 5Program in Biostatistics and Biomathematics, Fred Hutchinson Cancer Research Center, Seattle, WA, USA
  6. 6Providence Cancer Center, Providence Health and Services Oregon; and Departments of Medical Informatics and Clinical Epidemiology and Medicine, Oregon Health & Science University, Portland, OR, USA
  7. 7Department of Family Medicine, University of Vermont, Burlington, VT, USA
  8. 8Department of Family Medicine, Oregon Health & Science University, Portland, OR, USA
  9. 9Community and Family Medicine, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
  10. 10Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
  11. 11Department of Medicine, University of Washington School of Medicine, Seattle, WA, USA
  12. 12Department of Pathology; and UVM Cancer Center, University of Vermont, Burlington, VT, USA
  1. Correspondence to: J G Elmore jelmore{at}uw.edu
  • Accepted 15 May 2016

Abstract

Objective To evaluate the potential effect of second opinions on improving the accuracy of diagnostic interpretation of breast histopathology.

Design Simulation study.

Setting 12 different strategies for acquiring independent second opinions.

Participants Interpretations of 240 breast biopsy specimens by 115 pathologists, one slide for each case, compared with reference diagnoses derived by expert consensus.

Main outcome measures Misclassification rates for individual pathologists and for 12 simulated strategies for second opinions. Simulations compared accuracy of diagnoses from single pathologists with that of diagnoses based on pairing interpretations from first and second independent pathologists, where resolution of disagreements was by an independent third pathologist. 12 strategies were evaluated in which acquisition of second opinions depended on initial diagnoses, assessment of case difficulty or borderline characteristics, pathologists’ clinical volumes, or whether a second opinion was required by policy or desired by the pathologists. The 240 cases included benign without atypia (10% non-proliferative, 20% proliferative without atypia), atypia (30%), ductal carcinoma in situ (DCIS, 30%), and invasive cancer (10%). Overall misclassification rates and agreement statistics depended on the composition of the test set, which included a higher prevalence of difficult cases than in typical practice.

Results Misclassification rates significantly decreased (P<0.001) with all second opinion strategies except for the strategy limiting second opinions only to cases of invasive cancer. The overall misclassification rate decreased from 24.7% to 18.1% when all cases received second opinions (P<0.001). Obtaining both first and second opinions from pathologists with a high volume (≥10 breast biopsy specimens weekly) resulted in the lowest misclassification rate in this test set (14.3%, 95% confidence interval 10.9% to 18.0%). Obtaining second opinions only for cases with initial interpretations of atypia, DCIS, or invasive cancer decreased the over-interpretation of benign cases without atypia from 12.9% to 6.0%. Atypia cases had the highest misclassification rate after single interpretation (52.2%), remaining at more than 34% in all second opinion scenarios.

Conclusion Second opinions can statistically significantly improve diagnostic agreement for pathologists’ interpretations of breast biopsy specimens; however, variability in diagnosis will not be completely eliminated, especially for breast specimens with atypia.

Footnotes

  • The collection of data on cancer and vital status used in this study was supported in part by several state public health departments and cancer registries throughout the United States. For a full description of the Breast Cancer Surveillance Consortium sources see www.breastscreening.cancer.gov/work/acknowledgement.html.

  • Contributors: All authors contributed to the overall conception and design of the study. JE wrote the first draft of this manuscript. GL extracted the data. GL and MP performed the statistical analyses. All authors contributed to the interpretation of results and drafting of the manuscript. All authors read and approved the final manuscript. JE is the guarantor.

  • Funding: This work was supported by the National Cancer Institute (R01 CA140560, R01 CA172343, and K05 CA104699) and by the National Cancer Institute funded Breast Cancer Surveillance Consortium (HHSN261201100031C). The content is solely the responsibility of the authors and does not necessarily represent the views of the National Cancer Institute or the National Institutes of Health.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that all authors have support from the National Cancer Institute for the submitted work; no authors have relationships with any company that might have an interest in the submitted work in the previous three years; their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and no authors have non-financial interests that may be relevant to the submitted work.

  • Ethical approval: All study procedures were approved by the institutional review boards of Dartmouth College (No 21926), Fred Hutchinson Cancer Research Center (No 6958), Providence Health and Services of Oregon (No 10-055A), University of Vermont (No M09-281), and University of Washington (No 39631). All participating pathologists signed an informed consent form. Informed consent was not required of the women whose biopsy specimens were included.

  • Data sharing: Details of how to obtain additional data from the study (eg, statistical code, datasets) are available from the corresponding author at jelmore@uw.edu.

  • Transparency: The lead author (JE) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

View Full Text
Back to top