Lisa M. Kern, MD, MPH; Sameer Malhotra, MD, MA; Yolanda Barrón, MS; Jill Quaresimo, RN, JD; Rina Dhopeshwarkar, MPH; Michelle Pichardo, MPH; Alison M. Edwards, MStat; Rainu Kaushal, MD, MPH
Presented in part at the Annual Symposium of the American Medical Informatics Association, Washington, DC, 22–26 October 2011.
Note: The authors had full access to all of the data in the study and take responsibility for the integrity of the data and accuracy of the data analysis.
Acknowledgment: The authors thank Jonah Piascik for his assistance with data collection.
Grant Support: By the Agency for Healthcare Research and Quality (grant R18 HS 017067).
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M12-1178.
Reproducible Research Statement: Study protocol and statistical code: Available from Dr. Kern (e-mail, lmk2003@med.cornell.edu). Data set: Not available.
Requests for Single Reprints: Lisa M. Kern, MD, MPH, Department of Public Health, Weill Cornell Medical College, 425 East 61st Street, Suite 301, New York, NY; e-mail, lmk2003@med.cornell.edu.
Current Author Addresses: Drs. Kern and Kaushal and Ms. Edwards: Center for Healthcare Informatics and Policy, Weill Cornell Medical College, 425 East 61st Street, Suite 301, New York, NY 10065.
Dr. Malhotra: Weill Cornell Medical College, 575 Lexington Avenue, Box 110, New York, NY 10022.
Ms. Barrón: Center for Home Care and Research, Visiting Nurse Service of New York, 1250 Broadway, 20th Floor, New York, NY 10001.
Ms. Quaresimo: 4 Cleveland Drive, Poughkeepsie, NY 12601.
Ms. Dhopeshwarkar: 2665 Prosperity Avenue, Apartment 337, Fairfax, VA 22031.
Ms. Pichardo: Institute for Family Health, 22 West 19th Street, 8th Floor, New York, NY 10011.
Author Contributions: Conception and design: L.M. Kern, S. Malhotra, R. Kaushal.
Analysis and interpretation of the data: L.M. Kern, S. Malhotra, Y. Barrón, R. Dhopeshwarkar, M. Pichardo, A.M. Edwards, R. Kaushal.
Drafting of the article: L.M. Kern, S. Malhotra, M. Pichardo, R. Kaushal.
Critical revision of the article for important intellectual content: L.M. Kern, S. Malhotra, Y. Barrón, R. Dhopeshwarkar, R. Kaushal.
Final approval of the article: L.M. Kern, Y. Barrón, A.M. Edwards, R. Kaushal.
Provision of study materials or patients:
Statistical expertise: Y. Barrón, A.M. Edwards.
Obtaining of funding: L.M. Kern, R. Kaushal.
Administrative, technical, or logistic support: S. Malhotra, R. Dhopeshwarkar, M. Pichardo.
Collection and assembly of data: L.M. Kern, S. Malhotra, Y. Barrón, J. Quaresimo, M. Pichardo.
Chinese translation
The federal Electronic Health Record Incentive Program requires electronic reporting of quality from electronic health records, beginning in 2014. Whether electronic reports of quality are accurate is unclear.
To measure the accuracy of electronic reporting compared with manual review.
Cross-sectional study.
A federally qualified health center with a commercially available electronic health record.
All adult patients eligible in 2008 for 12 quality measures (using 8 unique denominators) were identified electronically. One hundred fifty patients were randomly sampled per denominator, yielding 1154 unique patients.
Receipt of recommended care, assessed by both electronic reporting and manual review. Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and absolute rates of recommended care were measured.
Sensitivity of electronic reporting ranged from 46% to 98% per measure. Specificity ranged from 62% to 97%, positive predictive value from 57% to 97%, and negative predictive value from 32% to 99%. Positive likelihood ratios ranged from 2.34 to 24.25 and negative likelihood ratios from 0.02 to 0.61. Differences between electronic reporting and manual review were statistically significant for 3 measures: Electronic reporting underestimated the absolute rate of recommended care for 2 measures (appropriate asthma medication [38% vs. 77%; P < 0.001] and pneumococcal vaccination [27% vs. 48%; P < 0.001]) and overestimated care for 1 measure (cholesterol control in patients with diabetes [57% vs. 37%; P = 0.001]).
This study addresses the accuracy of the measure numerator only.
Wide measure-by-measure variation in accuracy threatens the validity of electronic reporting. If variation is not addressed, financial incentives intended to reward high quality may not be given to the highest-quality providers.
Agency for Healthcare Research and Quality.
Kern LM, Malhotra S, Barrón Y, et al. Accuracy of Electronically Reported “Meaningful Use” Clinical Quality Measures: A Cross-sectional Study. Ann Intern Med. 2013;158:77–83. doi: https://doi.org/10.7326/0003-4819-158-2-201301150-00001
Download citation file:
© 2020
Published: Ann Intern Med. 2013;158(2):77-83.
DOI: 10.7326/0003-4819-158-2-201301150-00001
Healthcare Delivery and Policy.
Results provided by: