0
Clinical Guidelines |

High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions FREE

Douglas K. Owens, MD, MS; Amir Qaseem, MD, PhD, MHA; Roger Chou, MD; Paul Shekelle, MD, PhD, for the Clinical Guidelines Committee of the American College of Physicians
[+] Article and Author Information

Individuals who served on the Clinical Guidelines Committee from initiation of the project until its approval were Paul Shekelle, MD, PhD (Chair); Roger Chou, MD; Paul Dallas, MD; Thomas D. Denberg, MD, PhD; Nick Fitterman, MD; Mary Ann Forciea, MD; Robert H. Hopkins Jr., MD; Linda L. Humphrey, MD, MPH; Tanvir P. Mir, MD; Douglas K. Owens, MD, MS; Holger J. Schünemann, MD, PhD; Donna E. Sweet, MD; and David S. Weinberg, MD, MSc.


From Veterans Affairs Palo Alto Health Care System, Palo Alto, California; Stanford University, Stanford, California; American College of Physicians, Philadelphia, Pennsylvania; Oregon Health & Science University, Portland, Oregon; and West Los Angeles Medical Center, Los Angeles, California.


Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. government.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-1694.

Requests for Single Reprints: Amir Qaseem, MD, PhD, MHA, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106; e-mail, aqaseem@acponline.org.

Current Author Addresses: Dr. Owens: Stanford University, 117 Encina Commons, Stanford, CA 94305.

Dr. Qaseem: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.

Dr. Chou: Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Mail Code BICC, Portland, OR 97239.

Dr. Shekelle: West Los Angeles Veterans Affairs Medical Center, 11301 Wilshire Boulevard, Los Angeles, CA 90073.

Author Contributions: Conception and design: D.K. Owens, A. Qaseem, P. Shekelle.

Analysis and interpretation of the data: D.K. Owens, A. Qaseem.

Drafting of the article: D.K. Owens, A. Qaseem.

Critical revision for important intellectual content: D.K. Owens, A. Qaseem, R. Chou, P. Shekelle.

Final approval of the article: D.K. Owens, A. Qaseem, R. Chou, P. Shekelle.

Statistical expertise: D.K. Owens, A. Qaseem.

Obtaining of funding: A. Qaseem.

Administrative, technical, or logistic support: A. Qaseem.


Ann Intern Med. 2011;154(3):174-180. doi:10.7326/0003-4819-154-3-201102010-00007
Text Size: A A A

Health care costs in the United States are increasing unsustainably, and further efforts to control costs are inevitable and essential. Efforts to control expenditures should focus on the value, in addition to the costs, of health care interventions. Whether an intervention provides high value depends on assessing whether its health benefits justify its costs. High-cost interventions may provide good value because they are highly beneficial; conversely, low-cost interventions may have little or no value if they provide little benefit.

Thus, the challenge becomes determining how to slow the rate of increase in costs while preserving high-value, high-quality care. A first step is to decrease or eliminate care that provides no benefit and may even be harmful. A second step is to provide medical interventions that provide good value: medical benefits that are commensurate with their costs.

This article discusses 3 key concepts for understanding how to assess the value of health care interventions. First, assessing the benefits, harms, and costs of an intervention is essential to understand whether it provides good value. Second, assessing the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs that occur because the intervention was performed. Third, the incremental cost-effectiveness ratio estimates the additional cost required to obtain additional health benefits and provides a key measure of the value of a health care intervention.

Because health care costs are increasing unsustainably, further efforts to control costs are inevitable and essential. A crucial concern for patients, clinicians, and policymakers is whether it is possible to control costs while maintaining or improving the quality of care; that is, how should we undertake cost control? As part of our activities to develop clinical practice guidelines for the American College of Physicians (ACP), the Clinical Guidelines Committee plans to address these questions more directly with articles that highlight how clinicians can contribute to the delivery of high-value health care.

We believe that efforts to control expenditures should focus not on the costs or benefits alone but rather on the value of health care interventions The term value has other meanings (Appendix), but we use it as it is commonly understood,as an assessment of the benefit of an intervention relative to expenditures. Judgments about value are fundamental to decision making in most arenas but often have been missing in health care settings. The distinction between cost and value is critical: High-cost interventions may provide good value because they are highly beneficial; conversely, low-cost interventions may have little or no value if they provide little benefit.

We can group interventions broadly into 2 categories. The first category comprises interventions that provide minimal or no health benefit. These interventions typically have low value, regardless of their cost. For example, routine (as opposed to selective) imaging studies for low back pain meet these criteria (12) because such studies do not improve outcomes and may cause harm. Thus, this practice is wasteful and can be decreased or discontinued without a negative effect (and possibly with a positive effect) on the quality of care.

The second category comprises interventions that provide net benefit. Understanding the value of these interventions requires a quantitative assessment of their benefits and costs. Such assessments usually are done as cost-effectiveness analyses, which explicitly delineate the tradeoffs between health benefits and expenditures but do not determine whether an intervention should be offered. Determination of whether to offer an intervention depends on a judgment by the patient or policymaker about how much he or she is willing to pay for health benefits (3).

In this article, we briefly review why the delivery of high-value health care is important and then discuss the principles that can help clinicians and policymakers identify and deliver high-value care (4). We emphasize 3 key concepts. First, it is essential to assess the benefits, harms, and costs of an intervention to understand whether it provides good value. Second, an assessment of the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs that occur as a result of the intervention. Finally, the incremental cost-effectiveness ratio, which estimates how much additional cost is required to obtain additional health benefits, provides a key measure of the value of a health care intervention.

As the health care reform debate highlights, health care costs are high and are increasing at a rate that could potentially bankrupt the federal government and devastate family budgets. Thus, health care resources necessarily will be limited. An important rationale for the delivery of high-value health care is to preserve the delivery of interventions that do provide good value. Discussions of cost-effectiveness analysis, especially in the political discourse surrounding health care reform, often devolved into discussions about rationing, along with statements (presented without evidence) that using cost-effectiveness analyses would promote rationing. However, the term rationing as used in these debates implicitly refers to restricting the use of any intervention, regardless of its effectiveness or value.

If rationing is more appropriately defined as restricting the use of effective, high-value care, we believe that using cost-effectiveness analyses may help avoid rationing. That is, we believe that the best way to avoid inadvertent reductions in effective and efficient care is to identify and eliminate wasteful practices and to demonstrate which interventions provide high value.

Health care reform has focused attention on, and substantially increased funding for, comparative effectiveness research (CER). The Institute of Medicine defines CER as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care” (5) that identifies “what works best for which patients under what circumstances” (5). The purpose of CER is to help patients, providers, and policymakers make more informed decisions about health care. As some observers have commented, consumers often know more about the pros, cons, and costs of televisions, cars, and appliances than they do about health care interventions. If CER also assesses utilization and costs, it can further help provide a foundation for cost-effectiveness analysis (6).

Cost-effectiveness analysis compares the health benefits and costs of 2 or more preventive, diagnostic, or treatment strategies. The analysis estimates a measure of the value of one intervention compared with another, called the incremental cost-effectiveness ratio. This ratio measures efficiency and estimates the additional expenditures required to gain additional health benefits when a more effective and expensive strategy is undertaken. The incremental cost-effectiveness ratio is the change in costs divided by the change in health benefit when 2 strategies are compared (see the Appendix for more detail).

Health benefit can be measured in many ways, including conditions diagnosed or prevented or life-years or quality-adjusted life-years (QALYs) gained. A QALY is an important metric that is calculated by assessing how long a person lives and how persons assess quality of life during their lifetime (7). They are useful in measuring health benefits because they take into account the benefits from longer life or better quality of life. They are also useful because the benefits from cancer treatment, coronary artery bypass surgery, or screening for HIV can all be expressed in terms of QALYs. Thus, we can compare the cost-effectiveness of interventions for treating cancer or heart disease or of preventive interventions.

The cost-effectiveness ratio is expressed in dollars per health outcome, such as dollars per life-year gained, per infection prevented, or per condition diagnosed. For example, the cost-effectiveness of HIV screening could be expressed as $15 000 per QALY gained. Therefore, implementing HIV screening in a setting in which it was not done previously would cost $15 000 for each additional QALY gained from screening patients for and treating patients with HIV.

Cost-effectiveness analysis is a powerful tool but raises challenges in the evaluation of interventions. Evidence about effectiveness and harms may be of low quality or inconclusive. Costs can vary substantially among practice settings and may be difficult to obtain. Clinicians may lack data on how health outcomes affect patients, such as the quality of life of persons in specific health states. However, a carefully performed analysis highlights such issues and provides important insights about an intervention despite shortcomings of the available data.

Understanding Benefit and Harms: Will an Intervention Help or Hurt?

Understanding the benefits and harms of the intervention is central to assessing whether an intervention can provide high-value health care (811). Comparative effectiveness research provides the foundation for this assessment. Comparing strategies or therapies is particularly important, and CER directly addresses this comparison. Is one strategy to manage lipid levels better than another (12)? Does medical therapy or revascularization provide better outcomes in patients with ischemic heart disease (13)? Identifying the populations in whom an intervention is successful and whether effectiveness differs between particular subgroups also is important. For example, are percutaneous interventions better than coronary artery bypass surgery in patients with diabetes (1415)?

Different types of evidence may inform different aspects of comparative effectiveness. Randomized clinical trials in selected populations may provide an estimate of efficacy, defined as the impact of the intervention under ideal circumstances. However, to estimate efficacy, adherence to the intervention in the trial should be optimized to the extent possible; substantial rates of nonadherence will result in an underestimate of the efficacy of an intervention. Randomized trials performed in circumstances and populations that reflect how an intervention will be used in practice provide an estimate of effectiveness, defined as the impact of the intervention under typical, rather than ideal, circumstances.

Randomized trials or observational studies may provide data on harms. Nonadherence in a randomized trial could result in an underestimate of harms, because fewer patients would be exposed to the intervention. Because some harms occur only rarely, large observational studies may be the best source of evidence for such harms (11). Finally, in considering harms, both short-term harms (for example, an immediate adverse reaction to a drug) and long-term harms (for example, cancer induced by radiation from imaging studies [16]) are important to evaluate. The net benefit of an intervention is the extent to which the benefits outweigh the harms.

Systematic reviews are a mainstay of CER. Organizations that have led notable systematic reviews include the Evidence-based Practice Centers funded by the Agency for Healthcare Research and Quality (8), the Cochrane Collaboration, and the United Kingdom's National Institute for Health and Clinical Excellence. Systematic reviews and meta-analyses provide a comprehensive assessment of the evidence by reviewing all relevant studies. They provide the basis for clinical guidelines developed by the ACP (17).

Systems for grading the quality of evidence also are evolving (811, 1825), led by such organizations as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the U.S. Preventive Services Task Force; and the Agency for Healthcare Research and Quality (18). These systems help clinicians, patients, and policymakers understand how much confidence they can have in the estimates of harms and benefits from the available literature.

Understanding Costs: Can an Expensive Drug Cost Less Than an Inexpensive Drug?

The key principle in assessing costs is that the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs, defined as costs that occur as a result of the intervention. Downstream costs are included in this evaluation because they would not have occurred had the intervention not been done. For example, the cost of an HIV screening program includes the cost not only of HIV testing but also of treating HIV in patients in whom HIV is diagnosed through the screening program (2630).

Downstream costs may be very substantial, and an analyst may underestimate the cost of an intervention if downstream costs are not included in an assessment. In a program to screen surgeons for HIV to prevent transmission of HIV to patients, treatment costs for surgeons represented approximately 30% of the total cost of the program (31). Another example is implantation of an implantable cardioverter-defibrillator (ICD), which initially costs at least $30 000 to $50 000. Use of an ICD also costs approximately $6600 per year after implantation, as well as additional costs for periodic replacement of the generator every 5 to 8 years, which costs approximately $18 000 or more (32). Routine imaging for low back pain is expensive in part because it may lead to subsequent treatment costs from invasive procedures, including surgery (2).

In addition to downstream costs, downstream savings also should be considered when assessing the costs of an intervention. For example, consider patients with nonvalvular atrial fibrillation who are candidates for anticoagulation. Although use of warfarin involves ongoing monitoring costs, the cost-savings from reduction in strokes more than outweigh the costs of the drug and monitoring; therefore, use of warfarin reduces total costs in appropriate high-risk patients (3334). Warfarin also costs less than aspirin in high-risk patients because of its greater effectiveness. Thus, a drug that is more expensive (warfarin) can have lower long-term costs than a less-expensive drug (aspirin). This example demonstrates that appropriate evaluation of the cost of an intervention includes downstream costs and savings caused by the intervention, regardless of when these costs and savings occur.

The question of how to assess whether interventions that provide more benefits than harms provide high value occurs when one intervention is both more effective and more costly than an alternative intervention (29, 3538). For example, use of ICDs is more effective in preventing sudden cardiac death than medical management in selected patient populations (Tables 1 and 2). Whether the intervention is worth the additional expense is most commonly answered by estimating the incremental cost-effectiveness ratio (39), which is a measure of value. The ratio denotes how much additional money is required to gain additional health benefit when we move from a strategy that is less effective to a strategy that is more effective. Table 2 demonstrates how cost, benefit, and value can vary among interventions.

Table Jump PlaceholderTable 1.  Cost-Effectiveness Analysis of Implantable Cardioverter-Defibrillators
Table Jump PlaceholderTable 2.  Cost, Benefit, and Value of Medical Interventions

If an analysis demonstrates that an intervention is both better and less expensive than an alternative, we say that the intervention dominates the alternative. As previously noted, in patients who have nonvalvular atrial fibrillation and are at high risk for stroke, warfarin dominates aspirin because it is more effective and has lower total costs than aspirin (3334). Monitoring CD4 cell counts to manage HIV in resource-limited settings, such as South Africa, improves health outcomes and reduces long-term costs owing to reduced hospitalization for opportunistic infection and therefore dominates antiretroviral management by clinical symptoms alone (40). In both examples, an intervention with higher up-front costs reduces long-term costs because of downstream savings related to the intervention.

These concepts are demonstrated in the Figure, which shows 4 interventions represented by circles A through D. Intervention A dominates intervention D because intervention A is both more effective and less expensive than intervention D. Therefore, we would not choose intervention D if intervention A is a possibility. Intervention B is more effective than intervention A but is also more expensive; the incremental cost-effectiveness ratio of intervention B relative to intervention A is $25 000 per QALY gained. Thus, moving from intervention A to intervention B would cause additional health benefit, and each additional QALY gained would require $25 000 in additional expenditures. Intervention C is more effective and more expensive than intervention B and costs $65 000 per QALY gained compared with intervention B.

Grahic Jump Location
Figure.
Costs and benefits of 4 hypothetical interventions.

As the line between interventions becomes more horizontal, the cost-effectiveness ratio becomes less favorable because costs are increasing faster than benefits are. The slope of the line between 2 interventions represents the reciprocal of the cost-effectiveness ratio. A lower incremental cost-effectiveness ratio denotes more favorable cost-effectiveness. The lines between interventions A, B, and C are called the cost-effectiveness frontier. Any intervention with costs and QALYs below and to the right of the cost-effectiveness frontier would be dominated (such as intervention D), or less cost-effective than interventions on the frontier. QALY = quality-adjusted life-year.

Grahic Jump Location
The Cost-Effectiveness Threshold: How Much Is Health Worth?

Which intervention to choose in the Figure depends on the decision maker's cost-effectiveness threshold. That is, is the decision maker willing to pay $50 000, $100 000, or more for an additional QALY? The choice of a cost-effectiveness threshold is itself a value judgment by the decision maker and depends on several factors (3, 41), including who the decision maker is. Different decision makers may have different cost-effectiveness thresholds. A consumer who is trying to decide whether to pay for a more expensive drug may be more or less willing to pay for the drug than an insurance company, another payer, or the government would be.

If the government is the decision maker, the cost-effectiveness threshold would reflect the consensus preferences of the citizens. In the United States, no such consensus exists. In the United Kingdom, the National Health Service decides whether to pay for health interventions; cost-effectiveness analyses performed by the National Institute for Health and Clinical Excellence also often inform these decisions. The cost-effectiveness threshold reflected in these assessments is often between £20 000 and £30 000 (approximately $30 000 to $50 000) per QALY gained (42). The World Health Organization suggests that interventions that cost less than 3 times the per capita gross domestic product per disability-adjusted life-year gained are cost-effective and that interventions that cost less than the per capita gross domestic product are very cost-effective (43).

The choice of a particular decision maker's cost-effectiveness threshold depends on the decision maker's willingness to pay for better health outcomes, which in turn depends on how he or she values heath outcomes and money (3, 41). As evidenced by consumers' choices of differently priced health insurance plans, people value health differently. Garber and Phelps (41) used economic principles to demonstrate that, on the basis of plausible assumptions about values and risk attitudes, a cost-effectiveness threshold in the range of twice a person's annual income is reasonable. Another important determinant of a person's choice of a cost-effectiveness threshold is his or her attitude about risk (41). All else being equal, a person who is more averse to the risk for mortality and morbidity would be more willing to pay for health care and thus have a higher cost-effectiveness threshold. Garber and Phelps' analysis indicates that in a heterogeneous population (with varying values about health, money, and risk), use of a single cost-effectiveness threshold would cause some persons to receive more health care than they would choose and others to receive less.

The literature often cites a cost-effectiveness threshold of $50 000 per QALY gained (44). This value, which has neither theoretical nor empirical justification, was introduced in 1982 (45) and, if adjusted to current dollars, would be more than $120 000 per QALY gained. A recent analysis that evaluated the cost and benefits of modern health care in the United States found that people have been willing to pay for health care that costs approximately $109 000 per QALY or more (44), which is substantially higher than the $50 000 threshold. Whether this is a true reflection of societal preferences given the multiple factors that determine health care spending is not clear (46).

In general, most decision makers in the United States will conclude that interventions that cost less than $50 000 to $60 000 per QALY gained provide high value. However, the often-cited threshold of $50 000 per QALY gained may be lower than what many decision makers would choose. Our goal is not to specify a particular threshold, but to recognize that decision makers will need to choose a threshold that is consistent with their values and resources.

Best Practice Advice for Providing High-Value Health Care

A first step toward providing high-value health care is to decrease or discontinue the use of interventions that provide no benefit, such as routine rather than selective imaging in patients with low back pain (12). While developing guidelines for the ACP, we will highlight interventions that provide little or no benefit and, thus, are likely to be of low value.

A second step is to ensure that we provide interventions that are both effective and decrease costs, such as the use of warfarin in high-risk patients with nonvalvular atrial fibrillation. In the United Kingdom, the National Institute for Health and Clinical Excellence has identified numerous guidance recommendations that may decrease costs, such as better selection of antihypertensive drugs (47). A registry of cost-effectiveness analyses also contains interventions that are cost-saving (38).

For interventions that provide additional benefit at additional cost, we recommend assessing their value to patients and society by using cost-effectiveness analyses. Such analyses require specialized expertise and training, are often expensive, and thus are typically performed by investigators. The ACP has previously recommended the inclusion of cost-effectiveness analysis in CER (48), and we reiterate the importance of the development of such evidence: Evaluation of the cost-effectiveness of interventions provides important additional information for patients and clinicians.

We emphasize that the cost-effectiveness of an intervention should not solely determine its use. There may be ethical or justice-related reasons to provide interventions that do not achieve generally accepted levels of cost-effectiveness; such considerations may outweigh economic concerns. However, we argue that cost-effectiveness should be one factor among others that receives consideration. Higher-cost care does not always mean greater benefit for patients; therefore, we should focus on the value that care provides.

The unsustainable increase in health care costs will continue to cause profound changes in health care delivery. To preserve quality, we recommend careful assessment of both benefits and costs of interventions rather than focusing on either aspect alone. Evaluation of the effectiveness of interventions should include an analysis of both benefits and harms and use the best available evidence for each.

In assessing the costs of an intervention, we should consider its downstream costs and savings. For interventions that are better but cost more, we should evaluate their cost-effectiveness, an important approach to assessment of value. Considerations of equity or ethics may primarily determine an intervention's use in certain situations, but value will be highly relevant for many interventions. The successful delivery of high-value care will depend greatly on developing evidence to help understand which services provide good value and on engaging clinicians (49), policymakers, and patients in efforts to promote high-value care.

Chou R, Qaseem A, Snow V, Casey D, Cross JT Jr, Shekelle P, et al. Clinical Efficacy Assessment Subcommittee of the American College of Physicians.  Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med. 2007; 147:478-91.
PubMed
 
Chou R, Qaseem A, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  Diagnostic imaging for low back pain: advice for high-value health care from the American College of Physicians. Ann Intern Med. 2011; 154:181-9.
 
Owens DK.  Interpretation of cost-effectiveness analyses [Editorial]. J Gen Intern Med. 1998; 13:716-7.
PubMed
CrossRef
 
Fuchs VR.  New priorities for future biomedical innovations. N Engl J Med. 2010; 363:704-6.
PubMed
 
Institute of Medicine.  Initial National Priorities for Comparative Effectiveness Research. Accessed atwww.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspxon 4 May 2010.
 
Garber AM, Sox HC.  The role of costs in comparative effectiveness research. Health Aff (Millwood). 2010; 29:1805-11.
PubMed
 
Gold M, Siegel JE, Russell LB, Weinstein MC.  Cost-Effectiveness in Health and Medicine. New York: Oxford Univ Pr; 1996.
 
Slutsky J, Atkins D, Chang S, Sharp BA.  AHRQ series paper 1: comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:481-3.
PubMed
 
Helfand M, Balshem H.  AHRQ series paper 2: principles for developing guidance: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:484-90.
PubMed
 
Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N.  AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:491-501.
PubMed
 
Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P, Smith DH. et al.  AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:502-12.
PubMed
 
Sharma M, Ansari MT, Soares-Weiser K, Abousetta AM, Ooi TC, Sears M, et al; Agency for Healthcare Research and Quality.  Comparative Effectiveness of Lipid-Modifying Agents. Comparative Effectiveness Review No. 16. Published September 2009. Accessed atwww.effectivehealthcare.ahrq.gov/reports/final.cfmon 9 December 2010.
 
Boden WE, O'Rourke RA, Teo KK, Hartigan PM, Maron DJ, Kostuk WJ, et al. COURAGE Trial Research Group.  Optimal medical therapy with or without PCI for stable coronary disease. N Engl J Med. 2007; 356:1503-16.
PubMed
 
Bravata DM, Gienger AL, McDonald KM, Sundaram V, Perez MV, Varghese R. et al.  Systematic review: the comparative effectiveness of percutaneous coronary interventions and coronary artery bypass graft surgery. Ann Intern Med. 2007; 147:703-16.
PubMed
 
Hlatky MA, Boothroyd DB, Bravata DM, Boersma E, Booth J, Brooks MM. et al.  Coronary artery bypass surgery compared with percutaneous coronary interventions for multivessel disease: a collaborative analysis of individual patient data from ten randomised trials. Lancet. 2009; 373:1190-7.
PubMed
 
Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M, Gould R. et al.  Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch Intern Med. 2009; 169:2078-86.
PubMed
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153:194-9.
PubMed
 
Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB. et al.  AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010; 63:513-23.
PubMed
 
Brozek JL, Akl EA, Jaeschke R, Lang DM, Bossuyt P, Glasziou P, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies. Allergy. 2009; 64:1109-16.
PubMed
 
Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy. 2009; 64:669-77.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A, et al. GRADE Working Group.  Incorporating considerations of resources use into grading recommendations. BMJ. 2008; 336:1170-3.
PubMed
 
Schünemann HJ, Schünemann AH, Oxman AD, Brozek J, Glasziou P, Jaeschke R, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008; 336:1106-10.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. GRADE Working Group.  Going from evidence to recommendations. BMJ. 2008; 336:1049-51.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, GRADE Working Group.  What is “quality of evidence” and why is it important to clinicians? BMJ. 2008; 336:995-8.
PubMed
 
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE Working Group.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336:924-6.
PubMed
 
Sanders GD, Bayoumi AM, Sundaram V, Bilir SP, Neukermans CP, Rydzak CE. et al.  Cost-effectiveness of screening for HIV in the era of highly active antiretroviral therapy. N Engl J Med. 2005; 352:570-85.
PubMed
 
Paltiel AD, Weinstein MC, Kimmel AD, Seage GR 3rd, Losina E, Zhang H. et al.  Expanded screening for HIV in the United States—an analysis of cost-effectiveness. N Engl J Med. 2005; 352:586-95.
PubMed
 
Paltiel AD, Walensky RP, Schackman BR, Seage GR 3rd, Mercincavage LM, Weinstein MC. et al.  Expanded HIV screening in the United States: effect on clinical outcomes, HIV transmission, and costs. Ann Intern Med. 2006; 145:797-806.
PubMed
 
Sanders GD, Bayoumi AM, Holodniy M, Owens DK.  Cost-effectiveness of HIV screening in patients older than 55 years of age. Ann Intern Med. 2008; 148:889-903.
PubMed
 
Walensky RP, Freedberg KA, Weinstein MC, Paltiel AD.  Cost-effectiveness of HIV testing and treatment in the United States. Clin Infect Dis. 2007; 45:Suppl 4S248-54.
PubMed
 
Owens DK, Harris RA, Scott PM, Nease RF Jr.  Screening surgeons for HIV infection. A cost-effectiveness analysis. Ann Intern Med. 1995; 122:641-52.
PubMed
 
Sanders GD, Hlatky MA, Owens DK.  Cost-effectiveness of implantable cardioverter-defibrillators. N Engl J Med. 2005; 353:1471-80.
PubMed
 
Gage BF, Cardinalli AB, Albers GW, Owens DK.  Cost-effectiveness of warfarin and aspirin for prophylaxis of stroke in patients with nonvalvular atrial fibrillation. JAMA. 1995; 274:1839-45.
PubMed
 
Eckman MH, Rosand J, Greenberg SM, Gage BF.  Cost-effectiveness of using pharmacogenetic information in warfarin dosing for patients with nonvalvular atrial fibrillation. Ann Intern Med. 2009; 150:73-83.
PubMed
 
Brown AD, Garber AM.  Cost-effectiveness of 3 methods to enhance the sensitivity of Papanicolaou testing. JAMA. 1999; 281:347-53.
PubMed
 
Freedberg KA, Losina E, Weinstein MC, Paltiel AD, Cohen CJ, Seage GR. et al.  The cost effectiveness of combination antiretroviral therapy for HIV disease. N Engl J Med. 2001; 344:824-31.
PubMed
 
Khazeni N, Hutton DW, Garber AM, Hupert N, Owens DK.  Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009. Ann Intern Med. 2009; 151:829-39.
PubMed
 
The Center for the Evaluation of Value and Risk in Health.  Cost-Effectiveness Analysis (CEA) Registry. Accessed athttps://research.tufts-nemc.org/cear/default.aspxon 22 September 2010.
 
Detsky AS, Naglie IG.  A clinician's guide to cost-effectiveness analysis. Ann Intern Med. 1990; 113:147-54.
PubMed
 
Bendavid E, Young SD, Katzenstein DA, Bayoumi AM, Sanders GD, Owens DK.  Cost-effectiveness of HIV monitoring strategies in resource-limited settings: a southern African analysis. Arch Intern Med. 2008; 168:1910-8.
PubMed
 
Garber AM, Phelps CE.  Economic foundations of cost-effectiveness analysis. J Health Econ. 1997; 16:1-31.
PubMed
 
National Institute for Health and Clinical Excellence.  Guide to the Methods of Technology Appraisal. Published June 2008. Accessed atwww.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdfon 9 December 2010.
 
Hutubessy R, Chisholm D, Edejer TT.  Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Eff Resour Alloc. 2003; 1:8.
PubMed
 
Braithwaite RS, Meltzer DO, King JT Jr, Leslie D, Roberts MS.  What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care. 2008; 46:349-56.
PubMed
 
Ubel PA, Hirth RA, Chernew ME, Fendrick AM.  What is the price of life and why doesn't it increase at the rate of inflation? Arch Intern Med. 2003; 163:1637-41.
PubMed
 
Weinstein MC.  How much are Americans willing to pay for a quality-adjusted life year? [Editorial]. Med Care. 2008; 46:343-5.
PubMed
 
National Institute for Health and Clinical Excellence.  Cost saving guidance. Accessed atwww.nice.org.uk/usingguidance/benefitsofimplementation/costsavingguidance.jspon 22 September 2010.
 
American College of Physicians.  Information on cost-effectiveness: an essential product of a national comparative effectiveness program. Ann Intern Med. 2008; 148:956-61.
PubMed
 
Fuchs VR.  Eliminating “waste” in health care. JAMA. 2009; 302:2481-2.
PubMed
 
Appendix
Value

We use value to reflect an assessment of whether the benefits of an intervention justify its costs. Value is sometimes used in the decision-making and quality-of-life literature to refer to the importance that patients (or society) place on a health state.

Cost–Benefit Analysis

Cost–benefit analysis differs from cost-effectiveness analysis in that health outcomes are also expressed in dollars rather than in units of health. Thus, both the costs and the benefits of an intervention are expressed in dollars, and determining whether the benefits outweigh the costs is clear. Alternative methods are available for assessing health benefits in terms of dollars; however, because of the challenges inherent in assigning a monetary value to health outcomes, cost–benefit analysis has been used less frequently than cost-effectiveness analysis in health care settings.

Incremental Cost-Effectiveness Ratio

The incremental cost-effectiveness ratio is defined as the difference in costs divided by the difference in health benefit when 2 strategies are compared. For example, an analysis evaluating the cost-effectiveness of ICDs compared with medical therapy would use the following formula to determine the incremental cost-effectiveness ratio:

It is important to distinguish the incremental cost-effectiveness ratio from the average cost-effectiveness ratio; the latter is simply the cost of an intervention divided by the benefit. The average cost-effectiveness ratio can provide misleading estimates of the resources required to adopt a strategy and should not be used for decision making.

Dominance and Extended Dominance

The Figure demonstrates the concept of dominance. A dominated intervention is more expensive and less effective than another intervention. The Appendix Figure denotes another possible situation in which we show interventions A, B, C, and D as in the Figure, but we now also include intervention E, which is to the right and below the cost-effectiveness frontier.

Grahic Jump Location
Appendix Figure.
Extended dominance.

The cost-effectiveness ratios of interventions are shown on the lines between the interventions. Intervention E costs $35 000 per QALY gained relative to intervention A. Intervention B costs $15 000 per QALY gained relative to intervention E. Intervention D is dominated. Intervention E can be eliminated through extended dominance. QALY = quality-adjusted life-year.

Grahic Jump Location

In the Appendix Figure, intervention E is better than intervention A and more expensive but not as good or costly as intervention B. Unlike intervention D, which is dominated, no intervention is both better and less expensive than intervention E. In general, we would never choose intervention E; if our cost-effectiveness threshold were more than $35 000 per QALY gained, which would be necessary to choose intervention E over intervention A, we would always choose intervention B, which is better than intervention E and more cost effective ($25 000 per QALY gained rather than $35 000 per QALY gained). In this situation, we say that intervention E should not be chosen because we can eliminate it through extended dominance. In summary, we would not choose intervention D because intervention A is better and less expensive (strict dominance), and we would not choose intervention E because intervention B is better and has a more favorable cost-effectiveness ratio (extended dominance).

Figures

Grahic Jump Location
Figure.
Costs and benefits of 4 hypothetical interventions.

As the line between interventions becomes more horizontal, the cost-effectiveness ratio becomes less favorable because costs are increasing faster than benefits are. The slope of the line between 2 interventions represents the reciprocal of the cost-effectiveness ratio. A lower incremental cost-effectiveness ratio denotes more favorable cost-effectiveness. The lines between interventions A, B, and C are called the cost-effectiveness frontier. Any intervention with costs and QALYs below and to the right of the cost-effectiveness frontier would be dominated (such as intervention D), or less cost-effective than interventions on the frontier. QALY = quality-adjusted life-year.

Grahic Jump Location
Grahic Jump Location
Appendix Figure.
Extended dominance.

The cost-effectiveness ratios of interventions are shown on the lines between the interventions. Intervention E costs $35 000 per QALY gained relative to intervention A. Intervention B costs $15 000 per QALY gained relative to intervention E. Intervention D is dominated. Intervention E can be eliminated through extended dominance. QALY = quality-adjusted life-year.

Grahic Jump Location

Tables

Table Jump PlaceholderTable 1.  Cost-Effectiveness Analysis of Implantable Cardioverter-Defibrillators
Table Jump PlaceholderTable 2.  Cost, Benefit, and Value of Medical Interventions

Videos

In this video, Paul Shekelle, MD, PhD, offers additional insight into his clinical guideline, "High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions.

References

Chou R, Qaseem A, Snow V, Casey D, Cross JT Jr, Shekelle P, et al. Clinical Efficacy Assessment Subcommittee of the American College of Physicians.  Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med. 2007; 147:478-91.
PubMed
 
Chou R, Qaseem A, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  Diagnostic imaging for low back pain: advice for high-value health care from the American College of Physicians. Ann Intern Med. 2011; 154:181-9.
 
Owens DK.  Interpretation of cost-effectiveness analyses [Editorial]. J Gen Intern Med. 1998; 13:716-7.
PubMed
CrossRef
 
Fuchs VR.  New priorities for future biomedical innovations. N Engl J Med. 2010; 363:704-6.
PubMed
 
Institute of Medicine.  Initial National Priorities for Comparative Effectiveness Research. Accessed atwww.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspxon 4 May 2010.
 
Garber AM, Sox HC.  The role of costs in comparative effectiveness research. Health Aff (Millwood). 2010; 29:1805-11.
PubMed
 
Gold M, Siegel JE, Russell LB, Weinstein MC.  Cost-Effectiveness in Health and Medicine. New York: Oxford Univ Pr; 1996.
 
Slutsky J, Atkins D, Chang S, Sharp BA.  AHRQ series paper 1: comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:481-3.
PubMed
 
Helfand M, Balshem H.  AHRQ series paper 2: principles for developing guidance: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:484-90.
PubMed
 
Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N.  AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:491-501.
PubMed
 
Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P, Smith DH. et al.  AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63:502-12.
PubMed
 
Sharma M, Ansari MT, Soares-Weiser K, Abousetta AM, Ooi TC, Sears M, et al; Agency for Healthcare Research and Quality.  Comparative Effectiveness of Lipid-Modifying Agents. Comparative Effectiveness Review No. 16. Published September 2009. Accessed atwww.effectivehealthcare.ahrq.gov/reports/final.cfmon 9 December 2010.
 
Boden WE, O'Rourke RA, Teo KK, Hartigan PM, Maron DJ, Kostuk WJ, et al. COURAGE Trial Research Group.  Optimal medical therapy with or without PCI for stable coronary disease. N Engl J Med. 2007; 356:1503-16.
PubMed
 
Bravata DM, Gienger AL, McDonald KM, Sundaram V, Perez MV, Varghese R. et al.  Systematic review: the comparative effectiveness of percutaneous coronary interventions and coronary artery bypass graft surgery. Ann Intern Med. 2007; 147:703-16.
PubMed
 
Hlatky MA, Boothroyd DB, Bravata DM, Boersma E, Booth J, Brooks MM. et al.  Coronary artery bypass surgery compared with percutaneous coronary interventions for multivessel disease: a collaborative analysis of individual patient data from ten randomised trials. Lancet. 2009; 373:1190-7.
PubMed
 
Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M, Gould R. et al.  Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch Intern Med. 2009; 169:2078-86.
PubMed
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153:194-9.
PubMed
 
Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB. et al.  AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010; 63:513-23.
PubMed
 
Brozek JL, Akl EA, Jaeschke R, Lang DM, Bossuyt P, Glasziou P, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies. Allergy. 2009; 64:1109-16.
PubMed
 
Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy. 2009; 64:669-77.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A, et al. GRADE Working Group.  Incorporating considerations of resources use into grading recommendations. BMJ. 2008; 336:1170-3.
PubMed
 
Schünemann HJ, Schünemann AH, Oxman AD, Brozek J, Glasziou P, Jaeschke R, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008; 336:1106-10.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. GRADE Working Group.  Going from evidence to recommendations. BMJ. 2008; 336:1049-51.
PubMed
 
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, GRADE Working Group.  What is “quality of evidence” and why is it important to clinicians? BMJ. 2008; 336:995-8.
PubMed
 
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE Working Group.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336:924-6.
PubMed
 
Sanders GD, Bayoumi AM, Sundaram V, Bilir SP, Neukermans CP, Rydzak CE. et al.  Cost-effectiveness of screening for HIV in the era of highly active antiretroviral therapy. N Engl J Med. 2005; 352:570-85.
PubMed
 
Paltiel AD, Weinstein MC, Kimmel AD, Seage GR 3rd, Losina E, Zhang H. et al.  Expanded screening for HIV in the United States—an analysis of cost-effectiveness. N Engl J Med. 2005; 352:586-95.
PubMed
 
Paltiel AD, Walensky RP, Schackman BR, Seage GR 3rd, Mercincavage LM, Weinstein MC. et al.  Expanded HIV screening in the United States: effect on clinical outcomes, HIV transmission, and costs. Ann Intern Med. 2006; 145:797-806.
PubMed
 
Sanders GD, Bayoumi AM, Holodniy M, Owens DK.  Cost-effectiveness of HIV screening in patients older than 55 years of age. Ann Intern Med. 2008; 148:889-903.
PubMed
 
Walensky RP, Freedberg KA, Weinstein MC, Paltiel AD.  Cost-effectiveness of HIV testing and treatment in the United States. Clin Infect Dis. 2007; 45:Suppl 4S248-54.
PubMed
 
Owens DK, Harris RA, Scott PM, Nease RF Jr.  Screening surgeons for HIV infection. A cost-effectiveness analysis. Ann Intern Med. 1995; 122:641-52.
PubMed
 
Sanders GD, Hlatky MA, Owens DK.  Cost-effectiveness of implantable cardioverter-defibrillators. N Engl J Med. 2005; 353:1471-80.
PubMed
 
Gage BF, Cardinalli AB, Albers GW, Owens DK.  Cost-effectiveness of warfarin and aspirin for prophylaxis of stroke in patients with nonvalvular atrial fibrillation. JAMA. 1995; 274:1839-45.
PubMed
 
Eckman MH, Rosand J, Greenberg SM, Gage BF.  Cost-effectiveness of using pharmacogenetic information in warfarin dosing for patients with nonvalvular atrial fibrillation. Ann Intern Med. 2009; 150:73-83.
PubMed
 
Brown AD, Garber AM.  Cost-effectiveness of 3 methods to enhance the sensitivity of Papanicolaou testing. JAMA. 1999; 281:347-53.
PubMed
 
Freedberg KA, Losina E, Weinstein MC, Paltiel AD, Cohen CJ, Seage GR. et al.  The cost effectiveness of combination antiretroviral therapy for HIV disease. N Engl J Med. 2001; 344:824-31.
PubMed
 
Khazeni N, Hutton DW, Garber AM, Hupert N, Owens DK.  Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009. Ann Intern Med. 2009; 151:829-39.
PubMed
 
The Center for the Evaluation of Value and Risk in Health.  Cost-Effectiveness Analysis (CEA) Registry. Accessed athttps://research.tufts-nemc.org/cear/default.aspxon 22 September 2010.
 
Detsky AS, Naglie IG.  A clinician's guide to cost-effectiveness analysis. Ann Intern Med. 1990; 113:147-54.
PubMed
 
Bendavid E, Young SD, Katzenstein DA, Bayoumi AM, Sanders GD, Owens DK.  Cost-effectiveness of HIV monitoring strategies in resource-limited settings: a southern African analysis. Arch Intern Med. 2008; 168:1910-8.
PubMed
 
Garber AM, Phelps CE.  Economic foundations of cost-effectiveness analysis. J Health Econ. 1997; 16:1-31.
PubMed
 
National Institute for Health and Clinical Excellence.  Guide to the Methods of Technology Appraisal. Published June 2008. Accessed atwww.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdfon 9 December 2010.
 
Hutubessy R, Chisholm D, Edejer TT.  Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Eff Resour Alloc. 2003; 1:8.
PubMed
 
Braithwaite RS, Meltzer DO, King JT Jr, Leslie D, Roberts MS.  What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care. 2008; 46:349-56.
PubMed
 
Ubel PA, Hirth RA, Chernew ME, Fendrick AM.  What is the price of life and why doesn't it increase at the rate of inflation? Arch Intern Med. 2003; 163:1637-41.
PubMed
 
Weinstein MC.  How much are Americans willing to pay for a quality-adjusted life year? [Editorial]. Med Care. 2008; 46:343-5.
PubMed
 
National Institute for Health and Clinical Excellence.  Cost saving guidance. Accessed atwww.nice.org.uk/usingguidance/benefitsofimplementation/costsavingguidance.jspon 22 September 2010.
 
American College of Physicians.  Information on cost-effectiveness: an essential product of a national comparative effectiveness program. Ann Intern Med. 2008; 148:956-61.
PubMed
 
Fuchs VR.  Eliminating “waste” in health care. JAMA. 2009; 302:2481-2.
PubMed
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

High-Value, Cost-Conscious Health Care

The full report is titled “High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions.” It is in the 1 February 2011 issue of Annals of Internal Medicine (volume 154, pages 174-180). The authors are D.K. Owens, A. Qaseem, R. Chou, and P. Shekelle, for the Clinical Guidelines Committee of the American College of Physicians.

Read More...

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Topic Collections
PubMed Articles

Want to Subscribe?

Learn more about subscription options

Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)