Design and Use of Performance Measures to Decrease Low-Value Services and Achieve Cost-Conscious Care Free

Improving quality of care while controlling the cost of health care is a national priority. Several organizations that have traditionally focused on increasing use of beneficial services have intensified their efforts to decrease the use of low-value health care services. In 2006, the National Committee on Quality Assurance proposed a quality performance criterion for overuse of spine imaging (1). In 2008, the National Priorities Partnership identified “overuse” as 1 of 6 national health care priorities (2). More recently, the American College of Physicians launched its High-Value Care Initiative (3), which seeks to help physicians and patients understand the benefits, harms, and costs of interventions and whether services provide good value (4–6). For example, the American College of Physicians' paper on high-value care for low back pain advocates using diagnostic imaging only when patients have progressive neurologic deficits or signs or symptoms suggestive of a serious or specific underlying condition; routine imaging is otherwise considered to be low-value (4).

Just as we need performance measures to assess underuse of high-value services, we need valid, evidence-based measures of overuse. For example, at the same time that we should be measuring the proportion of patients aged 50 to 75 years who have been screened for colorectal cancer, we should be assessing the proportion of patients older than 75 years who had colorectal cancer screening that was not indicated. Performance measures for low-value services have the potential to be an important lever for changing clinician behavior through feedback, public reporting, clinical decision support, and financial incentives. This paper gives an overview of performance measures for low-value services, provides specific examples of possible measures to assess use of low-value services, and discusses how these measures can be used in clinical practice and policy.

This discussion includes two categories of interventions: 1) those for which the harms likely exceed the benefits and 2) those that may provide benefits, but for which a quantitative assessment of their benefits and costs by a multistakeholder group (patients, clinicians, and policymakers) suggests that the tradeoff between health benefits and expenditures is undesirable. Use of services for which the harms likely exceed the benefits has been defined by the Institute of Medicine as overuse. The RAND “appropriateness methodology” defines a test or treatment as “inappropriate” if the risk exceeds the benefit of the procedure for a specific indication. An example of such a service is colorectal cancer screening for patients older than 85 years; any small benefit in detecting polyps or early colorectal cancer is outweighed by the predictable and unavoidable possibility of harm from colonic perforation during the procedure and the competing risk for death from other causes (7).

The second category includes services for which the risk-to-benefit ratio is uncertain, as well as those that have a definable benefit but the benefit is judged to be outweighed by the relative harms and cost of the services (for example, by a cost-effectiveness analysis). There is no universally accepted methodology or bright line that defines the point at which a service has so little value that it should not be done (8); this will ultimately be a societal decision that depends on how much money we are willing to spend on health care, along with societal priorities. There are many examples of services with little or no value, such as screening for cervical cancer in low-risk women aged 65 years or older and in women who have had a total hysterectomy (uterus and cervix) for benign disease, performing imaging studies (rather than a high-sensitivity d-dimer measurement) as the initial diagnostic test in patients with low pretest probability of venous thromboembolism, and screening for chronic obstructive pulmonary disease with spirometry in individuals without respiratory symptoms (6).

A direct measure makes a judgment about whether an intervention was of low-value on the basis of the unique clinical circumstances of each eligible patient. For example, a direct measure of imaging for patients with acute low back pain would determine whether the patient had an imaging test that is typically of low-value and whether unusual circumstances justified the imaging test (such as a history of cancer). A clinician's performance is measured as the proportion of all eligible patients for whom he or she is responsible who received the low-value service and did not have extenuating circumstances. The theoretical optimal performance is 100% of patients not getting the test or intervention or having a documented justification for why they should get the service. However, direct measures require access to detailed clinical information to make these judgments, and these data are often not easily obtained.

An indirect measure evaluates use rates, and exceptionally high use rates are assumed to indicate that a provider (or provider group) frequently uses services of low-value. Indirect measures must be used when specific clinical criteria have not been defined to directly measure use of low-value services or when data sources containing the highly detailed clinical information required for direct measurement are not available. For example, administrative data and electronic health record data can be used to measure rates of diagnostic imaging for specific conditions. The ideal use rate of diagnostic imaging is not known because the data sources may be unable to identify all patients for whom an imaging study is actually justified and appropriate (such as patients with back pain who are at high risk for cancer or spinal infection). Indirect measures therefore use a normative approach and compare clinicians' use rates for a service to their peers. Because use data are ubiquitous and rates are relatively easy to calculate, indirect measures can assess many low-value services.

Many studies have supported the validity of relying on use rates as indirect measures of use of low-value services; these studies have shown wide variations in use of health care without apparent improvements in health outcomes (9–11). However, the assumption that very high rates represent more frequent use of low-value services may not always be true. If the number of people who truly need services varies substantially across clinicians, then raw rates of service use may not always be valid proxies for rates of unnecessary use. Thus, some caution is necessary. One study found that variations in use of coronary angiography, carotid endarterectomy, and upper gastrointestinal tract endoscopy across geographic areas were weakly associated with or not associated with rates of inappropriate use (12).

Interpretation of indirect measures is even more challenging when the proper use of a diagnostic test depends on the a priori likelihood of the disease being considered, and that probability may range from near 0 to almost 100%. For example, when is it appropriate to perform computed tomography to assess pulmonary emboli in a patient presenting with chest pain? At what a priori probability does the risk from radiation exposure exceed the likely benefit? Thus, even when normative data are available, use rates are often difficult to interpret in these circumstances. The Centers for Medicare & Medicaid Services Hospital Compare Web site reports hospitals' rates of follow-up mammography or ultrasonography within 45 days after screening mammography (13). To help patients interpret hospitals' rates, the site says, “A number that is much lower than 8% may mean there's not enough follow-up. A number much higher than 14% may mean there's too much unnecessary follow-up” (italics added). Whether outliers on this type of measures of use are truly overusing or underusing services is not clear and requires further study.

One potential way of improving gross use rates as indirect measures of low-value service use is to examine the rates at which results of diagnostic tests are determined to be negative (that is, no abnormality is found related to the presenting symptom). If a diagnostic test is used too often for low-risk patients, this will result in 1) a higher than expected rate of use and 2) a higher than normal rate of negative test results. For example, only one third of patients without known coronary artery disease were found to have obstructive lesions when they underwent elective cardiac catheterization (14). Similarly, a study of 28 177 patients who had revascularization found that 61% of patients with percutaneous coronary intervention and 51% of patients with coronary artery bypass grafting had testing for ischemia (most often nuclear imaging) by 24 months (15). However, of patients tested, only 11% had subsequent cardiac catheterization and only 5% had repeated revascularization (15). Thus, the a priori probability of obstruction was very low, suggesting that most persons tested had weak or no indications.

There are many other situations for which the rate of negative test results may be more helpful than gross utilization rates. For example, the Centers for Medicare & Medicaid Services Hospital Compare Quality Measures report rates of follow-up imaging after mammography. Some women have equivocal findings on mammography and require additional imaging to determine whether biopsy is needed. The proportion of women who need additional imaging after screening mammography depends on the case mix, especially if the center is a referral center that may be performing screening mammography in women with a history of breast cancer who are at higher risk for new lesions. However, the rate of additional imaging also depends on how risk-averse the interpreting radiologist is. A high rate of additional imaging coupled with a low rate of abnormal test results and subsequent biopsy would provide additional evidence of overuse.

Although using rates of negative test results as indirect measures of overuse has distinct advantages compared with using only crude rates of test ordering, changes in reporting standards would be needed for this to be possible. Administrative claims data lack the test results necessary to determine the rate of negative results. Even with electronic health records, test results are often stored as text rather than in discrete fields that could be queried. Nevertheless, this method holds promise for improving the validity and interpretability of using use rates as a way of measuring use of low-value services and should be prioritized for further evaluation.

Ideally, performance measures should be based on rigorous study designs (for example, randomized, controlled trials) that assessed the benefits, risks, and costs of interventions. However, to develop performance measures for low-value services, we will probably need to use data from different types of research design and methods, including subgroup analyses from clinical trials, cohort studies, cost–benefit analyses, and cost-effectiveness analyses. For example, a study reported that the net clinical benefit of anticoagulation among patients with atrial fibrillation and CHADS2 (congestive heart failure, hypertension, age >75 years, diabetes mellitus, and prior stroke) scores of 0 or 1 were “essentially zero” (16). This could be used to create a measure of anticoagulation use in this subgroup for whom anticoagulation has little or no value.

Performance measures for use of low-value services will probably need to be applied at the group level, such as a hospital or multispecialty group. Many individual clinicians may not see enough patients with the target conditions within the measurement interval to allow reliable measurement of differences in use. In addition, primary care physicians and specialists are often involved in decision making, and both should be held accountable rather than just the person who ordered the test. For example, some specialists may request that an imaging test be completed before they will see a patient. A gastroenterologist may recommend repeated colonoscopy for colonic polyp surveillance at a shorter interval than suggested by national guidelines, and a primary care physician may then feel obligated to follow that recommendation. Small-area variation studies have shown that regional and probably organizational cultures affect health service use.

However, applying these measures at the group level may be problematic if we consider the large number of physicians who are in solo or small-group practices. These physicians must refer into a hospital or imaging center for diagnostic testing. To combine their use rates with those of other unaffiliated physicians may be unfair, and the hospital itself may believe it is unfair to be held responsible for the use patterns of referring physicians. Nevertheless, group-level measures have the advantage of creating communities of clinicians with shared responsibility for decreasing use of low-value services.

Just as with other performance measures, those for low-value services can be used in a variety of ways to improve quality and health care value. A commonly used quality improvement strategy is audit and feedback, in which performance is measured and summaries of performance are given to clinicians (17). Audit and feedback seem to modestly improve quality of care, especially when performance is mediocre or poor (17). Less is known about the value of audit and feedback for measures of use of low-value services (18–20).

A second possible use is public reporting. The Centers for Medicare & Medicaid Services has continued to expand the number and type of public performance reports available over the Internet. This currently includes tools to compare hospitals, nursing homes, and dialysis centers, and plans are under way for providing information and tools to compare individual physicians (13, 21). However, little is known about whether public reporting of performance measures for use of low-value services will change use rates; such changes could occur if clinicians alter their practice patterns or if patients choose clinicians who seem to order services more judiciously (22). Previous studies suggest that patients are not familiar with public reports on quality (22, 23). More intensive dissemination efforts are needed if public reporting of overuse measures is to be effective. These will need to be coupled with patient education about the lack of need for specific services (such as imaging for low back pain and cervical cancer screening after a hysterectomy) and communicated through multiple media sources over a sustained period.

Financial incentives to discourage use of low-value services are also likely to be used (24). If direct measurement is possible, payors could deny payments for interventions for which patients do not meet specific criteria. For example, Medicare has established specific circumstances under which they will cover continuous positive airway pressure machines for patients with obstructive sleep apnea (25). Payors could also require high copayments from patients. When indirect measures are used (that is, when clinical indicators are not available or the data required are not accessible), payors could increase payments for clinicians with low rates of use of low-value services or decrease payments for clinicians with high rates. This is similar to other pay-for-performance programs. However, pay-for-performance programs that use indirect measures indiscriminately to address use of low-value services risk decreasing use so that some patients who need services do not receive them. For example, even though rates of coronary artery bypass grafting are lower in the United Kingdom, a substantial proportion of procedures are still judged to be inappropriate (26, 27). This raises concerns that efforts to decrease crude use rates without simultaneous efforts to increase appropriate use could be harmful.

Electronic health records with advanced clinical decision support (CDS) may also be used to decrease use of low-value services. The logic for direct measures of use of low-value services can be programmed into electronic health records to create point-of-care alerts and CDS tools to help clinicians decide whether they are about to order a low-value intervention. Electronic CDS reminders have been shown to improve performance for many, but not all, targets (28–31). Far fewer studies have examined whether CDS reminders and other tools can decrease unnecessary use. Most studies have addressed overprescribing antibiotics for upper respiratory tract infections (20). Two recent studies suggest that order-entry CDS tools can decrease inappropriate ordering of radiology tests and decrease the rate of growth of outpatient computed tomography (32, 33).

The first step in addressing the high cost of health care should be decreasing use of interventions that provide little or no benefit and are of low value. During the past 2 decades, we have learned a great deal about how to rigorously develop quality measures and how to use them to improve care. This knowledge can be applied to develop performance measures for low-value care. Measurements can be made directly, to evaluate care for individual patients, or indirectly, by analyzing use data at an aggregate level. Each measurement approach has strengths as well as limitations. Performance measures for low-value care need to be developed and tested with the same rigorous methods as performance measures for underuse of services; however, the evidence base used to develop measures will differ substantially. Evidence-based performance measures for low-value services can help motivate physicians to provide high-value care to their patients.

* This paper, written by David W. Baker, MD, MPH; Amir Qaseem, MD, PhD, MHA; P. Preston Reynolds, MD, PhD; Lea Anne Gardner, PhD, RN; and Eric C. Schneider, MD, MSc, was developed by the American College of Physicians Performance Measurement Committee: David W. Baker, MD, MPH (Chair); Mary Ann Forciea, MD; Sandra Adamson Fryhofer, MD; Robert A. Gluckman, MD; Catherine MacLean, MD, PhD; Nasseer A. Masoodi, MD, CMD, CP; Keith W. Michl, MD; P. Preston Reynolds, MD, PhD; and Nathan Spell, MD. Approved by the ACP Board of Regents on 14 February 2012.

This article was published at www.annals.org on 30 October 2012.