Does Pay-for-Performance Improve the Quality of Health Care? FREE

In the report Crossing the Quality Chasm (1), the Institute of Medicine called attention to the poor quality of health care in the United States. The Institute identified numerous factors contributing to poor quality, including the structure of the present health care payment system. The Institute found that, for certain clinical situations, health care payment arrangements may actually produce disincentives for quality.

The largely untapped potential of the health care payment system to change physician and health care system behavior has stimulated interest in both the scientific literature (2 - 3) and the popular press (4 - 6) for linking payments to performance on quality measures. Several health plans (3,7) and the Centers for Medicare & Medicaid Services are using explicit financial incentives for quality (8 - 9). The effectiveness of these programs has not been systematically evaluated, and despite enthusiasm about the potential for aligning financial incentives with high-quality health care, many fundamental questions about their optimal design, effectiveness, and implementation remain unanswered. For example, what types of clinical conditions or health care services should be the target of financial incentives to improve quality: chronic diseases, acute care, or preventive care services? How effective (and cost-effective) are financial incentives for quality? What are the optimum magnitude, frequency, and duration of financial incentives for quality? Should we reward achievement of an absolute threshold of performance, improvement over baseline performance, payment for each instance of a service regardless of the overall performance, or some combination of these? To whom should such incentives be directed: the patient (10), the health care provider, the provider group or hospital, or all of these parties? What types of quality measures should be rewarded: processes of care, outcomes, or both (11)? Are financial incentives for not providing inappropriate care (such as antibiotics for uncomplicated acute upper respiratory illnesses) effective? What is the optimum “package” of nonfinancial interventions (if any) to include with financial incentives for quality, for example, audit and feedback, recognition, clinical reminders, academic detailing, or information technology support (12 - 13)? Can we expect the effect of financial incentives to persist after they are stopped? Because any effective intervention will have some unanticipated effects, will important patient care activities that are not rewarded financially be neglected?

The purpose of this paper is to assess the relationship between explicit financial incentives and the provision of high-quality health care by systematically reviewing empirical studies. Because the evidence regarding the relationship between the financial incentives embedded in fee-for-service and capitation arrangements and the quality of health care has been thoroughly reviewed in previous work (14), we focused our review on literature that addresses explicit financial rewards for improving health care quality.

We conducted a systematic search of the English-language literature in PubMed to find articles published between 1 January 1980 and 14 November 2005 whose main objective was to assess the use of explicit financial incentives to improve health care quality. Our search algorithms combined Medical Subject Heading (MeSH) terms and text words. We used the following MeSH terms: quality of health care; insurance, health, reimbursement; physician incentive plans; and reimbursement, incentive. We designated words and word phrases as text word terms in our search algorithms to ensure that all words in the title, abstract, MeSH terms, and MeSH subheadings that matched our words were extracted from the electronic database (15). We indicated the following words and phrases as text word terms in our search: quality, quality of care, payment, payment system, reimbursement, risk adjustment, physicians, financial incentives, financing, incentive, health care, bonus, insurance, performance-based, and fees.

We reviewed additional publications found in bibliographies of retrieved articles, and we contacted experts about missing or unpublished studies. We included only English-language studies that reported original data. We were interested in identifying studies of explicit financial incentives directed at individual physicians and provider groups, as well as incentives at the level of the payment system, such as performance-based contracting. Eligible studies assessed the use of financial incentives as the independent variable and a measure of quality (such as immunization or cancer screening) as the dependent variable. Quality of care was defined as “the degree to which health care services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (16). The domains of quality were defined as access to care, structure of care, process of care, outcomes of care, and patient experience of care. Access to care is the patient's attainment of timely and appropriate health care. Structure of care is a feature of a health care organization or clinician relevant to the provision of health care. Process of care is a health care service provided to or on behalf of a patient. Outcome of care is a health state of a patient resulting from health care. Experience of care is the individual's or population's report concerning health care (17).

Studies were eligible for review if they were original reports providing empirical results and assessed the relationship between the explicit financial incentive and a quantitative measure of health care quality (17). Reviewers assessed articles in an unblinded, standardized manner (18 - 19). At least 2 authors reviewed the title and keywords of articles' PubMed citations to identify empirical studies. At least 2 authors then reviewed the abstracts of those studies to determine eligibility. All authors then abstracted and reviewed the full articles that addressed the objective of the review. When there were discrepancies, all authors met to discuss and reach consensus about article inclusion. There were no instances of disagreement in which consensus could not be reached.

We excluded eligible studies if there was no concurrent comparison group or if there was no baseline, preintervention analysis of the groups on the quality measure. Concurrent comparison groups are important because some studies of quality may show dramatic improvement over time but no statistically significant differences between intervention and comparison groups. We included randomized, controlled trials and controlled before-and-after studies. Because of the paucity of literature on this topic, we also included observational studies that, although not studying a specific intervention, nevertheless examined the relationship between financial incentives and quality in a cross-sectional analysis. We categorized the results of each study according to the effect of the financial incentive on the measure or measures of quality. Positive studies were those for which all measures of quality demonstrated a statistically significant improvement with the financial incentive. Studies with partial effects showed improved performance on some measures of quality but not others. Negative studies were those for which all measures of quality demonstrated a statistically significant decrease in quality with the financial incentive. The final category was for studies demonstrating no effect.

We used a checklist for methodologic quality published by Downs and Black (20). After we assessed the checklist for each article, a grade of 1 (poor) to 4 (excellent) was assigned to each included article. We did not use formal meta-analytic techniques because the included studies used many different measures of effect.

The Veterans Health Administration, the Robert Wood Johnson Foundation, the American Heart Association, and the National Institutes of Health supported this study but had no role in design, conduct, or reporting or in the decision to submit the paper for publication.

Our search for eligible studies is summarized in the Figure. Most of the articles were descriptions and not evaluations. Sixteen articles (21 - 36) that met the eligibility criteria were subsequently excluded because a concurrent comparison group was not analyzed or groups were not compared at baseline on the quality indicator (Table 1). The Appendix Table summarizes the 17 included studies. Two studies reported the effect of payment-system level financial incentives (37 - 38). The remaining 15 studies evaluated financial incentives directed to the provider group (39 - 47) or individual physician (48 - 53).

We found 2 studies of financial incentives provided at the payment system level (37 - 38). For example, Norton (37) studied the effectiveness of an incentive to improve access to health care for nursing home patients with debilitating acute and chronic conditions. The program included incentives to admit severely dependent patients, incentives for attainment of health status goals, and an incentive to discharge clinically appropriate patients. The intervention sites admitted statistically significantly more severely ill patients than nursing homes in the control group.

Despite the administrative and incentive costs of the program, the author's Markov model estimated an average cost savings to the Medicaid program of $3000 per nursing home stay over time. This was principally due to shorter stays. Therefore, the author asserted that the incentive and administrative costs were small compared with potential gains in improved health and lower overall health care expenditures.

Nine studies (39 - 47) evaluated the use of financial incentives directed to provider groups. Of these, 7 found partial or positive effects of financial incentives on measures of quality, although effect sizes of some studies were small. In 2 studies ((42), (44)), the provider group–level incentive resulted in a statistically significant improvement in the measure of quality of care (Appendix Table). In 5 other studies ((41), (43), (45)–(47)), the financial incentive had a partial effect. For example, Rosenthal and colleagues (47) found a small improvement in rates of cervical cancer screening between the intervention and comparison groups after the quality incentive program (difference, 3.6%; P = 0.02). Improvements in mammography screening rates and hemoglobin A_1c testing were not statistically significant. In 2 randomized trials (39 - 40), the group-level incentives for preventive health services had no effect compared with the control group.

Five of 6 studies ((48 - 50), (52 - 53)) found partial or positive effects of incentives directed at individual physicians. In a randomized trial, Fairbrother and colleagues (48) tested 2 types of incentives for immunizations in a pediatric population. Sixty physicians were randomly assigned to 1 of 3 intervention groups or a control group. The interventions were 1) bonus and feedback ($1000 for a 20% improvement from baseline, $2500 for a 40% improvement from baseline, and $5000 for reaching 80% up-to-date coverage regardless of baseline performance level); 2) enhanced fee-for-service and feedback ($5 for each vaccine administered within 30 days of its due date and $15 for each visit at which >1 vaccine was due and all due vaccines were administered); and 3) feedback only. The bonus group improved significantly in documented up-to-date immunization status, with an overall change of 25.3%, but none of the other groups improved statistically significantly compared with controls. However, with only 15 physicians per group, the power of the study was limited.

By the end of the study, more than two thirds of the physicians in the bonus group had improved enough to earn a bonus. Only 2 of the physicians in the enhanced fee-for-service group and 2 in the feedback-only group improved as much as those in the bonus group.

Shen (38) examined the effect of performance-based contracting on access to care among the most severely ill patients in a group being treated for substance abuse. Under performance-based contracting, achieving or failing to achieve predetermined quality measures affects funding in the following year. Providers not meeting the performance expectations had “special conditions” attached to the reimbursement contracts; in some cases, the subsequent contract period was shortened.

Shen hypothesized that the incentive (better pay for improving measured performance) would provide an unintended incentive to avoid the most severely ill patients, thereby restricting access to care for this group. This is termed adverse selection. Shen measured the proportion of outpatient clients classified as being the most severely ill before and after the introduction of performance-based contracting. The comparison group was Medicaid patients treated for substance abuse in the same years. Under performance-based contracting, the likelihood of a participant in the program being in the most severely ill group decreased (P ≤0.01), suggesting that adverse selection was occurring in response to the financial incentive.

In the article by Norton (37), the author noted that nursing homes had several strong incentives to “game the system,” that is, manipulate the program to increase payment. For example, there was an incentive for nursing homes to claim that they were admitting extremely disabled patients who then “miraculously” recovered over a short period. In both trials by Fairbrother and colleagues (48 - 49), the authors were careful to note that improvement was due primarily to improved documentation of up-to-date immunization status rather than actual vaccines given at the practice and that missed opportunities to vaccinate (that is, visits where vaccines were due but no vaccine was given) did not change. Roski and colleagues (41) examined the effect of bonus payments on identifying patients with tobacco use disorders and providing tobacco cessation advice in large multispecialty group practices. Similarly, the incentive was associated with an increased documentation of tobacco use status but not provision of advice to quit smoking. These studies again highlight the problem of gaming behavior, whereby the incentive produces improvements in documentation rather than a change in the quality of health care delivered to patients.

The issue of whether the incentive target should be designed as an absolute performance goal (that is, a defined threshold, such as 75% of patients with up-to-date immunization status), a relative performance goal (for example, 30% improvement from baseline), or a payment for each instance of a service regardless of the overall performance is an important question. We found 4 studies ((41 - 42), (47), (51)) that used an absolute performance target, 2 (39 - 40) that used relative performance targets, and 3 ((48 - 49), (53)) that used a combination of relative and absolute performance targets. Two studies showed that individuals or groups with the lowest baseline performance improved the most ((47), (53)); however, if threshold performance targets are used, they may garner the least performance pay (47).

In this systematic review, we found 17 eligible studies that addressed the question of whether explicit financial incentives improve the quality of health care (Appendix Table). Thirteen ((39 - 42), (44 - 49), (51 - 53)) of these examined process-of-care measures, most of which were for preventive services. Five studies ((37 - 38), (43), (47), (53)) assessed care of patients with chronic diseases. Only 6 studies (48 - 53), 2 by the same investigative team and assessing pediatric immunizations, assessed physician-level financial incentives, and only 2 (48 - 49) compared the type of incentive (bonus vs. enhanced fee-for-service). Five of 6 studies of physician-level financial incentives and 7 of 9 studies of provider group–level financial incentives found partial or positive effects of financial incentives on measures of quality. One of the 2 studies of incentives at the payment-system level found a positive effect on access to care, while the other showed evidence of gaming behavior or adverse selection, suggesting a negative effect on access to care. The cross-sectional surveys ((45 - 46), (50), (52)) gave some information about the relationship between incentives and quality, but generalizability was limited by lack of specificity about incentives as well as by the observational design. Four studies ((38), (41), (48 - 49)) suggested unintended effects of incentives. We found no studies evaluating the optimal duration of financial incentives for quality or persistence of their effects. We found only 1 study (37) that addressed cost-effectiveness and no studies that assessed incentives for not providing care (such as incentives for withholding antibiotics for simple upper respiratory illness or uncomplicated bacteriuria). Therefore, for several reasons, generalization from the existing studies is limited.

Despite these limitations, a few very preliminary conclusions can be drawn. Incentives require very careful design. Three studies showed that documentation, rather than actual use of the preventive service, improved statistically significantly with a financial incentive ((41), (48 - 49)). Shen (38) showed that adverse selection may have occurred with performance-based contracting in settings where providers can avoid sicker patients. These findings are important because they suggest that there is a response to incentives, although not necessarily the desired one. The challenge, then, is to design incentives with the intended goal in mind.

The observation that those with the lowest baseline performance may improve the most ((47), (53)) yet garner the smallest amount of performance pay if threshold performance targets are used (47) highlights the need to consider combined incentives for both overall improvement and achievement of a threshold. Policymakers should consider whether their goal is improving performance at the lower end of the spectrum, maintaining best performance, or both.

In designing performance measures for incentive programs, several issues should be noted. The best process-of-care measures are those for which evidence shows that better performance leads to better outcomes (1). Also, it is important to note that process-of-care measures may be more sensitive to quality differences than are measures of outcomes, because a poor outcome does not necessarily occur every time there is a quality problem. Therefore, one way to change behavior so that both quality and documentation improve may be to base the incentive on the combination of a process-of-care measure (for example, documentation of smoking cessation advice) and the outcome of interest (for example, tobacco quit rates). This approach may avoid the pitfalls of process-of-care measures alone that encourage gaming, as well as the disadvantage of basing incentives solely on outcomes that may be relatively rare or difficult to achieve and somewhat beyond the control of the provider. Thus, a combined approach capitalizes on the advantages and complementary nature of both types of quality-of-care measures.

Size of the bonus is probably also important. Possible explanations for the lack of effect or small effect in some studies may include the small size of the bonus ((39 - 40), (42), (51)). One qualitative study suggested that a bonus of at least 5% of a physician's capitation income may influence behavior (54). In contrast, the maximum bonus in the study by Grady and colleagues (51) was only $100. Similarly, when providers are paid by multiple insurers, the incentive may affect too few patients, effectively diluting the size of the incentive (40).

The last design issue to consider is that “end-of-year” compensation may not influence physician behavior as much as a concurrent fee or intermittent bonus. This is because lack of awareness of the intervention (40) and infrequent performance feedback seem to be substantial potential barriers to incentive effectiveness (41).

We cannot conclude from a single study that financial incentives are cost-effective. Norton (37) showed that using a combination of various types of incentives to improve both access to nursing home care and patient outcomes of nursing home care saved an estimated $3000 per stay in a Markov model. Because of the way the payment system is structured, however, these savings may not accrue to the Medicaid program that paid for the incentives, highlighting the importance of considering the “business case” (whether there is a return on investments made to improve quality) for quality improvement (55). Of course, more work in this area is urgently needed so that limited resources for improving health care quality can be targeted at the most effective interventions.

Most of the effect sizes of the provider group–level incentives were small (Appendix Table). This should not be surprising, because with hospital-level or provider group–level incentives, physicians cannot collect the full returns on their individual efforts to improve quality (56). Thus, the potential for some to “free-ride” on the efforts of others may reduce the efforts of all. Alternatively, the problem with rewarding physicians and not provider groups or hospitals is that the required institutional cooperation may not be present, implying that incentives are missing for an important element of the team delivering health care. For example, studies evaluating the chronic care model suggest that multidisciplinary teams produce better patient outcomes (57 - 59). Provider group–level or payment-system level incentives (if substantial enough) may provide the impetus to create infrastructure changes that are absent from traditional practice (60).

Most of the articles retrieved for this search were descriptions rather than evaluations. Our systematic review may be affected by negative publication bias because health care executives may have some disincentive to publish negative or ambiguous findings of pay-for-performance programs.

It seems intuitive that paying more money for higher-quality services will improve health care, but health care does not operate like a classic free market. The physician–patient dyad is a type of principal–agent relationship (61 - 63) from the economic literature on incentive contracts. Principal–agent theory addresses relationships where one individual (the patient) cannot directly observe or know the level of skill or effort expended by the other individual (the physician) doing the contracted work. Because patients do not have perfect knowledge of their medical condition, their need for care, or the expected outcome of health care services, they are willing to have physicians act as their agents in providing information and services. Because patients have asymmetric information about the need for and outcomes of health care, patient demand for health care may be unresponsive to technical quality. Therefore, one theoretical advantage of performance pay is that explicit financial incentives are provided even when patient demand for health care is unresponsive to quality. Physician effort in providing high quality is rewarded, regardless of whether patients recognize it.

It is important to note that financial incentives and the health care payment system have an important, although not exclusive, influence on the provision of quality. In economic terms, physicians are viewed as maximizing their utility function (56). (Utility can be defined as well-being.) Important factors included in the utility function, besides income, are professional and social status, altruistic concerns, the cost of the effort to provide care, and the uncertainty of the clinical effectiveness of treatment (64 - 66). It is generally accepted that professionals are motivated by the satisfaction of doing their jobs well (intrinsic motivation) (67). Indeed, it is doubtful whether some valued-but-difficult-to-observe dimensions of quality (such as empathy or listening in the medical encounter) would be provided at all if physicians were solely interested in income. Thus, physicians have both nonmonetary (that is, personal ethics, professional norms, regulatory control, clinical uncertainty) and monetary (from the payment system) incentives, all of which affect effort. This review addresses only the financial aspect of this complex issue and does not address the possibility that financial incentives may dilute physicians' intrinsic motivation.

Most physicians and hospitals are paid the same regardless of the quality of the health care they provide, producing no financial incentives for quality and, in some cases, disincentives. Thus, there is increasing enthusiasm for the idea of linking payment to performance. Despite widespread implementation, we found few informative studies of explicit financial incentives for quality. This literature review suggests some positive effects of financial incentives at the physician level, the provider group level, and the health care payment system level. The findings also suggest that ongoing monitoring of incentive programs is critical to determine whether incentives are having unintended effects on quality of care. A suggested research agenda for moving the field ahead is provided in Table 2 . Rigorous research, including randomized, controlled trials and observational studies with concurrent control groups, is needed to guide implementation of explicit financial incentives for health care quality and to assess their cost-effectiveness.