The full content of Annals is available to subscribers

Subscribe/Learn More  >
Original Research |

The Reliability of Medical Record Review for Estimating Adverse Event Rates

Eric J. Thomas, MD, MPH; Stuart R. Lipsitz, PhD; David M. Studdert, LLB, ScD, MPH; and Troyen A. Brennan, MD, JD, MPH
[+] Article, Author, and Disclosure Information

From Brigham and Women's Hospital and Harvard School of Public Health, Harvard University, Boston, Massachusetts; Medical University of South Carolina, Charleston, South Carolina; and University of Texas-Houston Medical School, Houston, Texas.

Grant Support: By the Robert Wood Johnson Foundation.

Requests for Single Reprints: Eric J. Thomas, MD, MPH, University of Texas-Houston Medical School, 6431 Fannin MSB 1.122, Houston, TX 77030.

Current Author Addresses: Dr. Thomas: University of Texas–Houston Medical School, 6431 Fannin MSB 1.122, Houston, TX 77030.

Dr. Brennan: Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115.

Dr. Studdert: Department of Health Policy and Management, Harvard School of Public Health, 677 Huntington Avenue, Boston, MA 02115.

Dr. Lipsitz: Department of Biometry and Epidemiology, Medical University of South Carolina, 135 Rutledge Avenue, Box 250551, Charleston, SC 29425.

Author Contributions: Conception and design: E.J. Thomas, S.R. Lipsitz, D.M. Studdert, T.A. Brennan.

Analysis and interpretation of the data: E.J. Thomas, S.R. Lipsitz, D.M. Studdert, T.A. Brennan.

Drafting of the article: E.J. Thomas, D.M. Studdert.

Critical revision of the article for important intellectual content: E.J. Thomas, T.A. Brennan.

Final approval of the article: E.J. Thomas.

Provision of study materials or patients: E.J. Thomas, T.A. Brennan.

Statistical expertise: S.R. Lipsitz.

Obtaining of funding: E.J. Thomas, T.A. Brennan.

Administrative, technical, or logistic support: E.J. Thomas.

Collection and assembly of data: E.J. Thomas, T.A. Brennan.

Ann Intern Med. 2002;136(11):812-816. doi:10.7326/0003-4819-136-11-200206040-00009
Text Size: A A A

Background: The data used by the U.S. Institute of Medicine to estimate deaths from medical errors come from a study that relied on nurse and physician reviews of medical records to detect the errors.

Objective: To measure the reliability of medical record review for detecting adverse events and negligent adverse events.

Design: Medical record review.

Setting: Hospitalizations in Utah and Colorado in 1992.

Measurements: After three independent reviews of 500 medical records, the following were measured: reliability and the effect of varying criteria for reviewer confidence in and reviewer agreement about the presence of adverse events.

Results: For agreements in judgments of adverse events among the three sets of reviews, the κ statistics ranged from 0.40 to 0.41 (95% CIs ranged from 0.30 to 0.51) for adverse events and from 0.19 to 0.23 (CIs, 0.05 to 0.37) for negligent adverse events. Rates for adverse events and for negligent adverse events varied substantially depending on the degree of agreement and the level of confidence that was required among reviewers.

Conclusion: Estimates of adverse event rates from medical record review, including those reported by the Institute of Medicine in its 2000 report on medical errors, are highly sensitive to the degree of consensus and confidence among reviewers.


Grahic Jump Location
Appendix Figure. *Forty-eight of the referred records were not reviewed by physicians.
Review sequence.
Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Submit a Comment/Letter

Summary for Patients

The Reliability of Medical Record Review for Estimating the Frequency of Medical Mistakes

The summary below is from the full report titled “The Reliability of Medical Record Review for Estimating Adverse Event Rates.” It is in the 4 June 2002 issue of Annals of Internal Medicine (volume 136, pages 812-816). The authors are EJ Thomas, SR Lipsitz, DM Studdert, and TA Brennan.


Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.