0
Editorials |

Self-regulation in the Era of Big Data: Appropriate Use of Appropriate Use CriteriaAppropriate Use of Appropriate Use Criteria FREE

Jacob A. Doll, MD; and Manesh R. Patel, MD
[+] Article, Author, and Disclosure Information

This article was published online first at www.annals.org on 10 March 2015.


From Duke University Medical Center, Durham, North Carolina.

Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M15-0418.

Requests for Single Reprints: Manesh R. Patel, MD, Associate Professor of Medicine, Duke University Medical Center, 2301 Erwin Road, DN7432, Durham, NC 27710; e-mail, manesh.patel@duke.edu.

Current Author Addresses: Dr. Doll: Duke University Medical Center, 2301 Erwin Road, DUMC 3845, Durham, NC 27710.

Dr. Patel: Associate Professor of Medicine, Duke University Medical Center, 2301 Erwin Road, DN7432, Durham, NC 27710.


Ann Intern Med. 2015;162(8):592-593. doi:10.7326/M15-0418
Text Size: A A A

At the core of the medical profession is the collaborative decision-making process between physician and patient, where clinical evidence and best practice guidelines are applied to optimize outcomes for the individual patient. This process has long been shielded from public view. Decisions on the use of cardiovascular procedures are of particular interest, in part because of wide practice variation (1). New information technologies, electronic health records, and large administrative databases now permit outside observation of clinical decision making. With observation comes the potential for regulation. Governments, payers, professional organizations, and patients all wish to influence the use of medical procedures, either to decrease costs or to increase quality. These stakeholders have increasing access to clinical information. Billing and clinical registry data are widely available. Patients can access their electronic health record online and post reviews of their physicians to social media. Use of “big data” to understand, quantify, and regulate clinical decision making is inevitable.

Appropriate use criteria (AUC) are tools intended to help interpret these data. Using a method initially developed by the RAND Corporation to address underuse in a rapidly changing health care system, the American College of Cardiology Foundation, in collaboration with other professional societies, established criteria to guide use of cardiovascular procedures, such as echocardiography, diagnostic catheterization, and revascularization (23). In the case of diagnostic catheterization, a panel rated 166 scenarios as appropriate, uncertain, or inappropriate on the basis of clinical trial evidence and expert clinical opinion (4). These criteria are intended to frame an individual decision between a physician and a patient but can play a key role in evaluating practice when applied to larger populations.

In this issue, Mohareb and colleagues (5) applied the 2012 AUC for diagnostic catheterization to a cohort of patients referred for angiography without a known history of coronary artery disease (CAD), with data collected from a registry encompassing 19 hospitals in Ontario, Canada. Overall, 58.2% of angiographic studies were rated appropriate, 31% were rated uncertain, and 10.8% were rated inappropriate, with substantial variation among hospitals in the percentage of studies rated appropriate. A stepwise decrease in diagnostic yield was noted for procedures rated appropriate (52.9%), uncertain (36.7%), or inappropriate (30.9%). Of note, studies rated inappropriate resulted in diagnosis of left main or triple-vessel disease in 7.1% of patients.

This study demonstrates some of the opportunities and challenges of applying AUC to large data sets. It used a registry with standard data elements that achieved universal capture of angiographic studies in Ontario. This permitted meaningful comparisons among hospitals. However, the registry lacked elements required to confidently assess appropriateness, notably whether chest pain was typical or atypical and whether stress testing categorized patients as intermediate-risk. Although reasonable adjustments were made to fit the AUC to available data, some misclassification probably remained. Poor clinical documentation and missing data can distort AUC findings, but this study, like similar analyses from the New York State database and the CathPCI Registry of the National Cardiovascular Data Registry, demonstrates that careful application of AUC to large data sets can generate findings of great interest to clinicians and policymakers (67).

The results of this study show that invasive angiography that is rated as appropriate is more likely to diagnose obstructive CAD. What about the 47% of procedures rated appropriate that did not find obstructive disease? It is important to recognize that not all indicated angiographic studies uncover CAD. Rather, the appropriateness rating indicates that the clinical scenario is one for which evidence supports a benefit of performing invasive angiography. A finding of no CAD in a patient with a high pretest probability, with resultant avoidance of unnecessary medications and further testing, is a valuable result. Similarly, the finding of obstructive disease among some patients with procedures rated inappropriate is expected. Performance of procedures rated inappropriate may be prompted by a unique clinical scenario not captured by the AUC or influenced by patient preference. Poor documentation or lack of cohesive medical systems providing upstream information could omit clinical data that would justify the procedure. These findings highlight the need for ongoing maintenance of AUC with an iterative process that incorporates new evidence from clinical trials and quality improvement initiatives.

Presently, AUC can affect care delivery in many ways. Individual clinicians should consider AUC when ordering a cardiovascular imaging test or catheterization. The American College of Cardiology Foundation and others are developing clinical decision support to help incorporate the AUC into practice. Hospitals should examine institutional and provider-level appropriateness to identify areas for improvement. The AUC could be integrated into electronic health records to prospectively identify potential areas of underuse and overuse.

However, further work is needed to take full advantage of big data to build a learning health care system in which AUC are seamlessly integrated into clinical care and policy. An example of such a system in development is the CathPCI Registry, which collects data elements necessary to assess appropriateness and provides confidential quarterly feedback to hospitals and physicians (8). The logical next step—use of AUC as a quality measure for public reporting and financial incentives—is more problematic. Some variables not represented in the AUC (such as extremes of age, comorbid conditions, and patient preference) may influence decision making, and further work is needed to understand how to identify outliers. Professional societies, regulators, and payers will need to standardize definitions so that AUC and performance measures can be applied uniformly and fairly.

There is broad interest in systems that use big data to assess appropriateness in order to reduce cost. However, that is a 1-sided approach to AUC. An ideal system would be evidence-based, use uniform and comprehensive clinical data, provide point-of-care decision support, and aim to improve quality by reducing overuse and underuse. The AUC could be the backbone of such a system and, if trusted by all stakeholders, could provide a practice-level alternative to preauthorization requirements or indiscriminant reductions in reimbursement. Physicians must embrace the opportunity for self-regulation that AUC offer to ensure that we remain advocates for our patients and stewards of our health system.

References

Wennberg DE. The Dartmouth Atlas of Cardiovascular Health Care. Chicago: AHA Pr; 1999.
 
Bonow RO, Douglas PS, Buxton AE, Cohen DJ, Curtis JP, Delong E, et al, American College of Cardiology Foundation. ACCF/AHA methodology for the development of quality measures for cardiovascular technology: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures. Circulation. 2011; 124:1483-502.
PubMed
CrossRef
 
Patel MR, Spertus JA, Brindis RG, Hendel RC, Douglas PS, Peterson ED, et al, American College of Cardiology Foundation. ACCF proposed method for evaluating the appropriateness of cardiovascular imaging. J Am Coll Cardiol. 2005; 46:1606-13.
PubMed
CrossRef
 
Patel MR, Bailey SR, Bonow RO, Chambers CE, Chan PS, Dehmer GJ, et al. ACCF/SCAI/AATS/AHA/ASE/ASNC/HFSA/HRS/SCCM/SCCT/SCMR/STS 2012 appropriate use criteria for diagnostic catheterization: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, Society for Cardiovascular Angiography and Interventions, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2012; 59:1995-2027.
PubMed
CrossRef
 
Mohareb MM, Qiu F, Cantor WJ, Kingsbury KJ, Ko DT, Wijeysundera HC. Validation of the appropriate use criteria for coronary angiography. A cohort study. Ann Intern Med. 2015; 162:549-556.
 
Bradley SM, Spertus JA, Kennedy KF, Nallamothu BK, Chan PS, Patel MR, et al. Patient selection for diagnostic coronary angiography and hospital-level percutaneous coronary intervention appropriateness: insights from the National Cardiovascular Data Registry. JAMA Intern Med. 2014; 174:1630-9.
PubMed
CrossRef
 
Hannan EL, Samadashvili Z, Cozzens K, Walford G, Holmes DR Jr, Jacobs AK, et al. Appropriateness of diagnostic catheterization for suspected coronary artery disease in New York State. Circ Cardiovasc Interv. 2014; 7:19-27.
PubMed
CrossRef
 
American College of Cardiology Foundation.  CathPCI Registry: Appropriate Use Criteria. Washington, DC: American College of Cardiology Foundation; 2014. Accessed at www.ncdr.com/webncdr/cathpci/home/auc on 13 February 2015.
 

Figures

Tables

References

Wennberg DE. The Dartmouth Atlas of Cardiovascular Health Care. Chicago: AHA Pr; 1999.
 
Bonow RO, Douglas PS, Buxton AE, Cohen DJ, Curtis JP, Delong E, et al, American College of Cardiology Foundation. ACCF/AHA methodology for the development of quality measures for cardiovascular technology: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures. Circulation. 2011; 124:1483-502.
PubMed
CrossRef
 
Patel MR, Spertus JA, Brindis RG, Hendel RC, Douglas PS, Peterson ED, et al, American College of Cardiology Foundation. ACCF proposed method for evaluating the appropriateness of cardiovascular imaging. J Am Coll Cardiol. 2005; 46:1606-13.
PubMed
CrossRef
 
Patel MR, Bailey SR, Bonow RO, Chambers CE, Chan PS, Dehmer GJ, et al. ACCF/SCAI/AATS/AHA/ASE/ASNC/HFSA/HRS/SCCM/SCCT/SCMR/STS 2012 appropriate use criteria for diagnostic catheterization: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, Society for Cardiovascular Angiography and Interventions, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2012; 59:1995-2027.
PubMed
CrossRef
 
Mohareb MM, Qiu F, Cantor WJ, Kingsbury KJ, Ko DT, Wijeysundera HC. Validation of the appropriate use criteria for coronary angiography. A cohort study. Ann Intern Med. 2015; 162:549-556.
 
Bradley SM, Spertus JA, Kennedy KF, Nallamothu BK, Chan PS, Patel MR, et al. Patient selection for diagnostic coronary angiography and hospital-level percutaneous coronary intervention appropriateness: insights from the National Cardiovascular Data Registry. JAMA Intern Med. 2014; 174:1630-9.
PubMed
CrossRef
 
Hannan EL, Samadashvili Z, Cozzens K, Walford G, Holmes DR Jr, Jacobs AK, et al. Appropriateness of diagnostic catheterization for suspected coronary artery disease in New York State. Circ Cardiovasc Interv. 2014; 7:19-27.
PubMed
CrossRef
 
American College of Cardiology Foundation.  CathPCI Registry: Appropriate Use Criteria. Washington, DC: American College of Cardiology Foundation; 2014. Accessed at www.ncdr.com/webncdr/cathpci/home/auc on 13 February 2015.
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment/Letter
Considerations of the Appropriate Use Criteria in “Big Data” Analysis
Posted on May 4, 2015
Carlos Polanco, Ph.D., (*, a) Jorge Alberto Castañón González, M.D., (b) Vladimir N. Uversky, Ph.D., (c)
(a) Universidad Nacional Autónoma de México (b) Hospital Juárez de Mexico (c) University of South Florida
Conflict of Interest: None Declared

We read with the interest the editorial by Doll and Patel emphasizing the interpretation of "big data" through appropriate use criteria (AUC) tools, to unveil the process of clinical decision-making and outcomes that has long been shielded from public view. (1). Although we agree with the authors that the use of big data to understand, quantify, and regulate clinical decision-making is a desired goal, the inherent limitations of the algorithms exploited in the AUC programming for the effective search lies in the fact that the possible future stages of the algorithm would be similar to some past stage (known), but some will not because some variables will not be represented in the AUC, or will have confounding variables as those gathered while treating patients with comorbid conditions, or those who presented problems that might be confounding and contradictory, characterized by imperfect, inconsistent, or even inaccurate information (2). In this sense, storage, and processing of "big data" relating to clinical decisions, is not a technological problem, but a semantic one; i.e., the correct formulation of the question. The paradoxes raised in the mathematical discipline named Theory of Sets (3) result in an erroneous exposition of the questions.

 

Sincerely,

Carlos Polanco, Ph.D., (*, a)

Jorge Alberto Castañón González, M.D., (b)

Vladimir N. Uversky, Ph.D., (c)

(a) Department of Mathematics, Universidad Nacional Autónoma de México, México City, México.

(b) Department of Critical Care Unit and Biomedical Research, Hospital Juárez de México, México City, México.

(c) Department of Molecular Medicine, University of South Florida, Tampa Florida, USA.

 

References

(1) Doll JA, Patel MR. Self-regulation in the Era of Big Data: Appropriate Use of Appropriate Use Criteria. Annals of Internal Medicine 2015 162(8):592 DOI: 10.7326/M15-0418.

(2) Hoffman S, Podgurski A The use and misuse of biomedical data: is bigger really better? Am J Law med 2013;39(4):497-53.

(3) Cronen VE, Johnson KM, Lannamann JW. Paradoxes, double binds, and reflexive loops: an alternative theoretical perspective. Fam Process 1982 Mar;21(1):91-112.

Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Journal Club
Topic Collections
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)