Skip Navigation
American College of Physicians Logo
  • Subscribe
  • Submit a Manuscript
  • Sign In
    Sign in below to access your subscription for full content
    INDIVIDUAL SIGN IN
    Sign In|Set Up Account
    You will be directed to acponline.org to register and create your Annals account
    INSTITUTIONAL SIGN IN
    Open Athens|Shibboleth|Log In
    Annals of Internal Medicine
    SUBSCRIBE
    Subscribe to Annals of Internal Medicine.
    You will be directed to acponline.org to complete your purchase.
Annals of Internal Medicine Logo Menu
  • Latest
  • Issues
  • Channels
  • CME/MOC
  • In the Clinic
  • Journal Club
  • Web Exclusives
  • Author Info
Advanced Search
  • ‹ PREV ARTICLE
  • This Issue
  • NEXT ARTICLE ›
Clinical Guidelines |1 February 2011

High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions Free

Douglas K. Owens, MD, MS; Amir Qaseem, MD, PhD, MHA; Roger Chou, MD; Paul Shekelle, MD, PhD; for the Clinical Guidelines Committee of the American College of Physicians

Douglas K. Owens, MD, MS

Amir Qaseem, MD, PhD, MHA

Roger Chou, MD

Paul Shekelle, MD, PhD

for the Clinical Guidelines Committee of the American College of Physicians

Article, Author, and Disclosure Information
Author, Article, and Disclosure Information
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. government.
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-1694.
Requests for Single Reprints: Amir Qaseem, MD, PhD, MHA, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106; e-mail, aqaseem@acponline.org.
Current Author Addresses: Dr. Owens: Stanford University, 117 Encina Commons, Stanford, CA 94305.
Dr. Qaseem: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.
Dr. Chou: Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Mail Code BICC, Portland, OR 97239.
Dr. Shekelle: West Los Angeles Veterans Affairs Medical Center, 11301 Wilshire Boulevard, Los Angeles, CA 90073.
Author Contributions: Conception and design: D.K. Owens, A. Qaseem, P. Shekelle.
Analysis and interpretation of the data: D.K. Owens, A. Qaseem.
Drafting of the article: D.K. Owens, A. Qaseem.
Critical revision for important intellectual content: D.K. Owens, A. Qaseem, R. Chou, P. Shekelle.
Final approval of the article: D.K. Owens, A. Qaseem, R. Chou, P. Shekelle.
Statistical expertise: D.K. Owens, A. Qaseem.
Obtaining of funding: A. Qaseem.
Administrative, technical, or logistic support: A. Qaseem.
Individuals who served on the Clinical Guidelines Committee from initiation of the project until its approval were Paul Shekelle, MD, PhD (Chair); Roger Chou, MD; Paul Dallas, MD; Thomas D. Denberg, MD, PhD; Nick Fitterman, MD; Mary Ann Forciea, MD; Robert H. Hopkins Jr., MD; Linda L. Humphrey, MD, MPH; Tanvir P. Mir, MD; Douglas K. Owens, MD, MS; Holger J. Schünemann, MD, PhD; Donna E. Sweet, MD; and David S. Weinberg, MD, MSc.
  • From Veterans Affairs Palo Alto Health Care System, Palo Alto, California; Stanford University, Stanford, California; American College of Physicians, Philadelphia, Pennsylvania; Oregon Health & Science University, Portland, Oregon; and West Los Angeles Medical Center, Los Angeles, California.
×
  • ‹ PREV ARTICLE
  • This Issue
  • NEXT ARTICLE ›
Jump To
  • Full Article
  • FULL ARTICLE
  • FULL ARTICLE
    • Abstract
    • Rationale for High-Value Care: High Costs and Unsustainable Cost Increases
    • Comparative Effectiveness and Cost-Effectiveness
    • Interpreting Cost-Effectiveness Analysis: What Is High-Value Care?
    • Conclusion
      1. References
    • Appendix
  • Figures
  • Tables
  • Supplements
  • Audio/Video
  • Summary for Patients
  • Clinical Slide Sets
  • CME / MOC
  • Comments
  • Twitter Link
  • Facebook Link
  • Email Link
More
  • LinkedIn Link

Abstract

Health care costs in the United States are increasing unsustainably, and further efforts to control costs are inevitable and essential. Efforts to control expenditures should focus on the value, in addition to the costs, of health care interventions. Whether an intervention provides high value depends on assessing whether its health benefits justify its costs. High-cost interventions may provide good value because they are highly beneficial; conversely, low-cost interventions may have little or no value if they provide little benefit.

Thus, the challenge becomes determining how to slow the rate of increase in costs while preserving high-value, high-quality care. A first step is to decrease or eliminate care that provides no benefit and may even be harmful. A second step is to provide medical interventions that provide good value: medical benefits that are commensurate with their costs.

This article discusses 3 key concepts for understanding how to assess the value of health care interventions. First, assessing the benefits, harms, and costs of an intervention is essential to understand whether it provides good value. Second, assessing the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs that occur because the intervention was performed. Third, the incremental cost-effectiveness ratio estimates the additional cost required to obtain additional health benefits and provides a key measure of the value of a health care intervention.

Because health care costs are increasing unsustainably, further efforts to control costs are inevitable and essential. A crucial concern for patients, clinicians, and policymakers is whether it is possible to control costs while maintaining or improving the quality of care; that is, how should we undertake cost control? As part of our activities to develop clinical practice guidelines for the American College of Physicians (ACP), the Clinical Guidelines Committee plans to address these questions more directly with articles that highlight how clinicians can contribute to the delivery of high-value health care.
We believe that efforts to control expenditures should focus not on the costs or benefits alone but rather on the value of health care interventions The term value has other meanings (7), but we use it as it is commonly understood,as an assessment of the benefit of an intervention relative to expenditures. Judgments about value are fundamental to decision making in most arenas but often have been missing in health care settings. The distinction between cost and value is critical: High-cost interventions may provide good value because they are highly beneficial; conversely, low-cost interventions may have little or no value if they provide little benefit.
We can group interventions broadly into 2 categories. The first category comprises interventions that provide minimal or no health benefit. These interventions typically have low value, regardless of their cost. For example, routine (as opposed to selective) imaging studies for low back pain meet these criteria (1, 2) because such studies do not improve outcomes and may cause harm. Thus, this practice is wasteful and can be decreased or discontinued without a negative effect (and possibly with a positive effect) on the quality of care.
The second category comprises interventions that provide net benefit. Understanding the value of these interventions requires a quantitative assessment of their benefits and costs. Such assessments usually are done as cost-effectiveness analyses, which explicitly delineate the tradeoffs between health benefits and expenditures but do not determine whether an intervention should be offered. Determination of whether to offer an intervention depends on a judgment by the patient or policymaker about how much he or she is willing to pay for health benefits (3).
In this article, we briefly review why the delivery of high-value health care is important and then discuss the principles that can help clinicians and policymakers identify and deliver high-value care (4). We emphasize 3 key concepts. First, it is essential to assess the benefits, harms, and costs of an intervention to understand whether it provides good value. Second, an assessment of the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs that occur as a result of the intervention. Finally, the incremental cost-effectiveness ratio, which estimates how much additional cost is required to obtain additional health benefits, provides a key measure of the value of a health care intervention.

Rationale for High-Value Care: High Costs and Unsustainable Cost Increases

As the health care reform debate highlights, health care costs are high and are increasing at a rate that could potentially bankrupt the federal government and devastate family budgets. Thus, health care resources necessarily will be limited. An important rationale for the delivery of high-value health care is to preserve the delivery of interventions that do provide good value. Discussions of cost-effectiveness analysis, especially in the political discourse surrounding health care reform, often devolved into discussions about rationing, along with statements (presented without evidence) that using cost-effectiveness analyses would promote rationing. However, the term rationing as used in these debates implicitly refers to restricting the use of any intervention, regardless of its effectiveness or value.
If rationing is more appropriately defined as restricting the use of effective, high-value care, we believe that using cost-effectiveness analyses may help avoid rationing. That is, we believe that the best way to avoid inadvertent reductions in effective and efficient care is to identify and eliminate wasteful practices and to demonstrate which interventions provide high value.

Comparative Effectiveness and Cost-Effectiveness

Health care reform has focused attention on, and substantially increased funding for, comparative effectiveness research (CER). The Institute of Medicine defines CER as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care” (5) that identifies “what works best for which patients under what circumstances” (5). The purpose of CER is to help patients, providers, and policymakers make more informed decisions about health care. As some observers have commented, consumers often know more about the pros, cons, and costs of televisions, cars, and appliances than they do about health care interventions. If CER also assesses utilization and costs, it can further help provide a foundation for cost-effectiveness analysis (6).
Cost-effectiveness analysis compares the health benefits and costs of 2 or more preventive, diagnostic, or treatment strategies. The analysis estimates a measure of the value of one intervention compared with another, called the incremental cost-effectiveness ratio. This ratio measures efficiency and estimates the additional expenditures required to gain additional health benefits when a more effective and expensive strategy is undertaken. The incremental cost-effectiveness ratio is the change in costs divided by the change in health benefit when 2 strategies are compared (see the 7 for more detail).
Health benefit can be measured in many ways, including conditions diagnosed or prevented or life-years or quality-adjusted life-years (QALYs) gained. A QALY is an important metric that is calculated by assessing how long a person lives and how persons assess quality of life during their lifetime (7). They are useful in measuring health benefits because they take into account the benefits from longer life or better quality of life. They are also useful because the benefits from cancer treatment, coronary artery bypass surgery, or screening for HIV can all be expressed in terms of QALYs. Thus, we can compare the cost-effectiveness of interventions for treating cancer or heart disease or of preventive interventions.
The cost-effectiveness ratio is expressed in dollars per health outcome, such as dollars per life-year gained, per infection prevented, or per condition diagnosed. For example, the cost-effectiveness of HIV screening could be expressed as $15 000 per QALY gained. Therefore, implementing HIV screening in a setting in which it was not done previously would cost $15 000 for each additional QALY gained from screening patients for and treating patients with HIV.
Cost-effectiveness analysis is a powerful tool but raises challenges in the evaluation of interventions. Evidence about effectiveness and harms may be of low quality or inconclusive. Costs can vary substantially among practice settings and may be difficult to obtain. Clinicians may lack data on how health outcomes affect patients, such as the quality of life of persons in specific health states. However, a carefully performed analysis highlights such issues and provides important insights about an intervention despite shortcomings of the available data.

Understanding Benefit and Harms: Will an Intervention Help or Hurt?

Understanding the benefits and harms of the intervention is central to assessing whether an intervention can provide high-value health care (8–11). Comparative effectiveness research provides the foundation for this assessment. Comparing strategies or therapies is particularly important, and CER directly addresses this comparison. Is one strategy to manage lipid levels better than another (12)? Does medical therapy or revascularization provide better outcomes in patients with ischemic heart disease (13)? Identifying the populations in whom an intervention is successful and whether effectiveness differs between particular subgroups also is important. For example, are percutaneous interventions better than coronary artery bypass surgery in patients with diabetes (14, 15)?
Different types of evidence may inform different aspects of comparative effectiveness. Randomized clinical trials in selected populations may provide an estimate of efficacy, defined as the impact of the intervention under ideal circumstances. However, to estimate efficacy, adherence to the intervention in the trial should be optimized to the extent possible; substantial rates of nonadherence will result in an underestimate of the efficacy of an intervention. Randomized trials performed in circumstances and populations that reflect how an intervention will be used in practice provide an estimate of effectiveness, defined as the impact of the intervention under typical, rather than ideal, circumstances.
Randomized trials or observational studies may provide data on harms. Nonadherence in a randomized trial could result in an underestimate of harms, because fewer patients would be exposed to the intervention. Because some harms occur only rarely, large observational studies may be the best source of evidence for such harms (11). Finally, in considering harms, both short-term harms (for example, an immediate adverse reaction to a drug) and long-term harms (for example, cancer induced by radiation from imaging studies [16]) are important to evaluate. The net benefit of an intervention is the extent to which the benefits outweigh the harms.
Systematic reviews are a mainstay of CER. Organizations that have led notable systematic reviews include the Evidence-based Practice Centers funded by the Agency for Healthcare Research and Quality (8), the Cochrane Collaboration, and the United Kingdom's National Institute for Health and Clinical Excellence. Systematic reviews and meta-analyses provide a comprehensive assessment of the evidence by reviewing all relevant studies. They provide the basis for clinical guidelines developed by the ACP (17).
Systems for grading the quality of evidence also are evolving (8–11, 18–25), led by such organizations as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the U.S. Preventive Services Task Force; and the Agency for Healthcare Research and Quality (18). These systems help clinicians, patients, and policymakers understand how much confidence they can have in the estimates of harms and benefits from the available literature.

Understanding Costs: Can an Expensive Drug Cost Less Than an Inexpensive Drug?

The key principle in assessing costs is that the cost of an intervention should include not only the cost of the intervention itself but also any downstream costs, defined as costs that occur as a result of the intervention. Downstream costs are included in this evaluation because they would not have occurred had the intervention not been done. For example, the cost of an HIV screening program includes the cost not only of HIV testing but also of treating HIV in patients in whom HIV is diagnosed through the screening program (26–30).
Downstream costs may be very substantial, and an analyst may underestimate the cost of an intervention if downstream costs are not included in an assessment. In a program to screen surgeons for HIV to prevent transmission of HIV to patients, treatment costs for surgeons represented approximately 30% of the total cost of the program (31). Another example is implantation of an implantable cardioverter-defibrillator (ICD), which initially costs at least $30 000 to $50 000. Use of an ICD also costs approximately $6600 per year after implantation, as well as additional costs for periodic replacement of the generator every 5 to 8 years, which costs approximately $18 000 or more (32). Routine imaging for low back pain is expensive in part because it may lead to subsequent treatment costs from invasive procedures, including surgery (2).
In addition to downstream costs, downstream savings also should be considered when assessing the costs of an intervention. For example, consider patients with nonvalvular atrial fibrillation who are candidates for anticoagulation. Although use of warfarin involves ongoing monitoring costs, the cost-savings from reduction in strokes more than outweigh the costs of the drug and monitoring; therefore, use of warfarin reduces total costs in appropriate high-risk patients (33, 34). Warfarin also costs less than aspirin in high-risk patients because of its greater effectiveness. Thus, a drug that is more expensive (warfarin) can have lower long-term costs than a less-expensive drug (aspirin). This example demonstrates that appropriate evaluation of the cost of an intervention includes downstream costs and savings caused by the intervention, regardless of when these costs and savings occur.

Interpreting Cost-Effectiveness Analysis: What Is High-Value Care?

The question of how to assess whether interventions that provide more benefits than harms provide high value occurs when one intervention is both more effective and more costly than an alternative intervention (29, 35–38). For example, use of ICDs is more effective in preventing sudden cardiac death than medical management in selected patient populations (Tables 1 and 2). Whether the intervention is worth the additional expense is most commonly answered by estimating the incremental cost-effectiveness ratio (39), which is a measure of value. The ratio denotes how much additional money is required to gain additional health benefit when we move from a strategy that is less effective to a strategy that is more effective. Table 2 demonstrates how cost, benefit, and value can vary among interventions.

Table 1.

Cost-Effectiveness Analysis of Implantable Cardioverter-Defibrillators

Table 1.

Table 2.

Cost, Benefit, and Value of Medical Interventions

Table 2.
If an analysis demonstrates that an intervention is both better and less expensive than an alternative, we say that the intervention dominates the alternative. As previously noted, in patients who have nonvalvular atrial fibrillation and are at high risk for stroke, warfarin dominates aspirin because it is more effective and has lower total costs than aspirin (33, 34). Monitoring CD4 cell counts to manage HIV in resource-limited settings, such as South Africa, improves health outcomes and reduces long-term costs owing to reduced hospitalization for opportunistic infection and therefore dominates antiretroviral management by clinical symptoms alone (40). In both examples, an intervention with higher up-front costs reduces long-term costs because of downstream savings related to the intervention.
These concepts are demonstrated in the Figure, which shows 4 interventions represented by circles A through D. Intervention A dominates intervention D because intervention A is both more effective and less expensive than intervention D. Therefore, we would not choose intervention D if intervention A is a possibility. Intervention B is more effective than intervention A but is also more expensive; the incremental cost-effectiveness ratio of intervention B relative to intervention A is $25 000 per QALY gained. Thus, moving from intervention A to intervention B would cause additional health benefit, and each additional QALY gained would require $25 000 in additional expenditures. Intervention C is more effective and more expensive than intervention B and costs $65 000 per QALY gained compared with intervention B.
Figure.
Costs and benefits of 4 hypothetical interventions.

As the line between interventions becomes more horizontal, the cost-effectiveness ratio becomes less favorable because costs are increasing faster than benefits are. The slope of the line between 2 interventions represents the reciprocal of the cost-effectiveness ratio. A lower incremental cost-effectiveness ratio denotes more favorable cost-effectiveness. The lines between interventions A, B, and C are called the cost-effectiveness frontier. Any intervention with costs and QALYs below and to the right of the cost-effectiveness frontier would be dominated (such as intervention D), or less cost-effective than interventions on the frontier. QALY = quality-adjusted life-year.

The Cost-Effectiveness Threshold: How Much Is Health Worth?

Which intervention to choose in the Figure depends on the decision maker's cost-effectiveness threshold. That is, is the decision maker willing to pay $50 000, $100 000, or more for an additional QALY? The choice of a cost-effectiveness threshold is itself a value judgment by the decision maker and depends on several factors (3, 41), including who the decision maker is. Different decision makers may have different cost-effectiveness thresholds. A consumer who is trying to decide whether to pay for a more expensive drug may be more or less willing to pay for the drug than an insurance company, another payer, or the government would be.
If the government is the decision maker, the cost-effectiveness threshold would reflect the consensus preferences of the citizens. In the United States, no such consensus exists. In the United Kingdom, the National Health Service decides whether to pay for health interventions; cost-effectiveness analyses performed by the National Institute for Health and Clinical Excellence also often inform these decisions. The cost-effectiveness threshold reflected in these assessments is often between £20 000 and £30 000 (approximately $30 000 to $50 000) per QALY gained (42). The World Health Organization suggests that interventions that cost less than 3 times the per capita gross domestic product per disability-adjusted life-year gained are cost-effective and that interventions that cost less than the per capita gross domestic product are very cost-effective (43).
The choice of a particular decision maker's cost-effectiveness threshold depends on the decision maker's willingness to pay for better health outcomes, which in turn depends on how he or she values heath outcomes and money (3, 41). As evidenced by consumers' choices of differently priced health insurance plans, people value health differently. Garber and Phelps (41) used economic principles to demonstrate that, on the basis of plausible assumptions about values and risk attitudes, a cost-effectiveness threshold in the range of twice a person's annual income is reasonable. Another important determinant of a person's choice of a cost-effectiveness threshold is his or her attitude about risk (41). All else being equal, a person who is more averse to the risk for mortality and morbidity would be more willing to pay for health care and thus have a higher cost-effectiveness threshold. Garber and Phelps' analysis indicates that in a heterogeneous population (with varying values about health, money, and risk), use of a single cost-effectiveness threshold would cause some persons to receive more health care than they would choose and others to receive less.
The literature often cites a cost-effectiveness threshold of $50 000 per QALY gained (44). This value, which has neither theoretical nor empirical justification, was introduced in 1982 (45) and, if adjusted to current dollars, would be more than $120 000 per QALY gained. A recent analysis that evaluated the cost and benefits of modern health care in the United States found that people have been willing to pay for health care that costs approximately $109 000 per QALY or more (44), which is substantially higher than the $50 000 threshold. Whether this is a true reflection of societal preferences given the multiple factors that determine health care spending is not clear (46).
In general, most decision makers in the United States will conclude that interventions that cost less than $50 000 to $60 000 per QALY gained provide high value. However, the often-cited threshold of $50 000 per QALY gained may be lower than what many decision makers would choose. Our goal is not to specify a particular threshold, but to recognize that decision makers will need to choose a threshold that is consistent with their values and resources.

Best Practice Advice for Providing High-Value Health Care

A first step toward providing high-value health care is to decrease or discontinue the use of interventions that provide no benefit, such as routine rather than selective imaging in patients with low back pain (1, 2). While developing guidelines for the ACP, we will highlight interventions that provide little or no benefit and, thus, are likely to be of low value.
A second step is to ensure that we provide interventions that are both effective and decrease costs, such as the use of warfarin in high-risk patients with nonvalvular atrial fibrillation. In the United Kingdom, the National Institute for Health and Clinical Excellence has identified numerous guidance recommendations that may decrease costs, such as better selection of antihypertensive drugs (47). A registry of cost-effectiveness analyses also contains interventions that are cost-saving (38).
For interventions that provide additional benefit at additional cost, we recommend assessing their value to patients and society by using cost-effectiveness analyses. Such analyses require specialized expertise and training, are often expensive, and thus are typically performed by investigators. The ACP has previously recommended the inclusion of cost-effectiveness analysis in CER (48), and we reiterate the importance of the development of such evidence: Evaluation of the cost-effectiveness of interventions provides important additional information for patients and clinicians.
We emphasize that the cost-effectiveness of an intervention should not solely determine its use. There may be ethical or justice-related reasons to provide interventions that do not achieve generally accepted levels of cost-effectiveness; such considerations may outweigh economic concerns. However, we argue that cost-effectiveness should be one factor among others that receives consideration. Higher-cost care does not always mean greater benefit for patients; therefore, we should focus on the value that care provides.

Conclusion

The unsustainable increase in health care costs will continue to cause profound changes in health care delivery. To preserve quality, we recommend careful assessment of both benefits and costs of interventions rather than focusing on either aspect alone. Evaluation of the effectiveness of interventions should include an analysis of both benefits and harms and use the best available evidence for each.
In assessing the costs of an intervention, we should consider its downstream costs and savings. For interventions that are better but cost more, we should evaluate their cost-effectiveness, an important approach to assessment of value. Considerations of equity or ethics may primarily determine an intervention's use in certain situations, but value will be highly relevant for many interventions. The successful delivery of high-value care will depend greatly on developing evidence to help understand which services provide good value and on engaging clinicians (49), policymakers, and patients in efforts to promote high-value care.

References

  1. Chou
    R
    ,  
    Qaseem
    A
    ,  
    Snow
    V
    ,  
    Casey
    D
    ,  
    Cross
    JT
    Jr
    ,  
    Shekelle
    P
    .  et al. 
    Clinical Efficacy Assessment Subcommittee of the American College of Physicians
    Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society.
    Ann Intern Med
    2007
    147
    478
    91
     PubMed
  2. Chou
    R
    ,  
    Qaseem
    A
    ,  
    Owens
    DK
    ,  
    Shekelle
    P
    .  
    Clinical Guidelines Committee of the American College of Physicians
    Diagnostic imaging for low back pain: advice for high-value health care from the American College of Physicians.
    Ann Intern Med
    2011
    154
    181
    9
  3. Owens
    DK
    .  
    Interpretation of cost-effectiveness analyses [Editorial].
    J Gen Intern Med
    1998
    13
    716
    7
     PubMed
    CrossRef
  4. Fuchs
    VR
    .  
    New priorities for future biomedical innovations.
    N Engl J Med
    2010
    363
    704
    6
     PubMed
  5. Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. Accessed atwww.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspxon 4 May 2010.
  6. Garber
    AM
    ,  
    Sox
    HC
    .  
    The role of costs in comparative effectiveness research.
    Health Aff (Millwood)
    2010
    29
    1805
    11
     PubMed
  7. Gold
    M
    ,  
    Siegel
    JE
    ,  
    Russell
    LB
    ,  
    Weinstein
    MC
    .  
    Cost-Effectiveness in Health and Medicine.
    New York
    Oxford Univ Pr
    1996
  8. Slutsky
    J
    ,  
    Atkins
    D
    ,  
    Chang
    S
    ,  
    Sharp
    BA
    .  
    AHRQ series paper 1: comparing medical interventions: AHRQ and the effective health-care program.
    J Clin Epidemiol
    2010
    63
    481
    3
     PubMed
  9. Helfand
    M
    ,  
    Balshem
    H
    .  
    AHRQ series paper 2: principles for developing guidance: AHRQ and the effective health-care program.
    J Clin Epidemiol
    2010
    63
    484
    90
     PubMed
  10. Whitlock
    EP
    ,  
    Lopez
    SA
    ,  
    Chang
    S
    ,  
    Helfand
    M
    ,  
    Eder
    M
    ,  
    Floyd
    N
    .  
    AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program.
    J Clin Epidemiol
    2010
    63
    491
    501
     PubMed
  11. Chou
    R
    ,  
    Aronson
    N
    ,  
    Atkins
    D
    ,  
    Ismaila
    AS
    ,  
    Santaguida
    P
    ,  
    Smith
    DH
    .  et al. 
    AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program.
    J Clin Epidemiol
    2010
    63
    502
    12
     PubMed
  12. Sharma M, Ansari MT, Soares-Weiser K, Abousetta AM, Ooi TC, Sears M, et al; Agency for Healthcare Research and Quality. Comparative Effectiveness of Lipid-Modifying Agents. Comparative Effectiveness Review No. 16. Published September 2009. Accessed atwww.effectivehealthcare.ahrq.gov/reports/final.cfmon 9 December 2010.
  13. Boden
    WE
    ,  
    O'Rourke
    RA
    ,  
    Teo
    KK
    ,  
    Hartigan
    PM
    ,  
    Maron
    DJ
    ,  
    Kostuk
    WJ
    .  et al. 
    COURAGE Trial Research Group
    Optimal medical therapy with or without PCI for stable coronary disease.
    N Engl J Med
    2007
    356
    1503
    16
     PubMed
  14. Bravata
    DM
    ,  
    Gienger
    AL
    ,  
    McDonald
    KM
    ,  
    Sundaram
    V
    ,  
    Perez
    MV
    ,  
    Varghese
    R
    .  et al. 
    Systematic review: the comparative effectiveness of percutaneous coronary interventions and coronary artery bypass graft surgery.
    Ann Intern Med
    2007
    147
    703
    16
     PubMed
  15. Hlatky
    MA
    ,  
    Boothroyd
    DB
    ,  
    Bravata
    DM
    ,  
    Boersma
    E
    ,  
    Booth
    J
    ,  
    Brooks
    MM
    .  et al. 
    Coronary artery bypass surgery compared with percutaneous coronary interventions for multivessel disease: a collaborative analysis of individual patient data from ten randomised trials.
    Lancet
    2009
    373
    1190
    7
     PubMed
  16. Smith-Bindman
    R
    ,  
    Lipson
    J
    ,  
    Marcus
    R
    ,  
    Kim
    KP
    ,  
    Mahesh
    M
    ,  
    Gould
    R
    .  et al. 
    Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer.
    Arch Intern Med
    2009
    169
    2078
    86
     PubMed
  17. Qaseem
    A
    ,  
    Snow
    V
    ,  
    Owens
    DK
    ,  
    Shekelle
    P
    .  
    Clinical Guidelines Committee of the American College of Physicians
    The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods.
    Ann Intern Med
    2010
    153
    194
    9
     PubMed
  18. Owens
    DK
    ,  
    Lohr
    KN
    ,  
    Atkins
    D
    ,  
    Treadwell
    JR
    ,  
    Reston
    JT
    ,  
    Bass
    EB
    .  et al. 
    AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program.
    J Clin Epidemiol
    2010
    63
    513
    23
     PubMed
  19. Brozek
    JL
    ,  
    Akl
    EA
    ,  
    Jaeschke
    R
    ,  
    Lang
    DM
    ,  
    Bossuyt
    P
    ,  
    Glasziou
    P
    .  et al. 
    GRADE Working Group
    Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies.
    Allergy
    2009
    64
    1109
    16
     PubMed
  20. Brozek
    JL
    ,  
    Akl
    EA
    ,  
    Alonso-Coello
    P
    ,  
    Lang
    D
    ,  
    Jaeschke
    R
    ,  
    Williams
    JW
    .  et al. 
    GRADE Working Group
    Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions.
    Allergy
    2009
    64
    669
    77
     PubMed
  21. Guyatt
    GH
    ,  
    Oxman
    AD
    ,  
    Kunz
    R
    ,  
    Jaeschke
    R
    ,  
    Helfand
    M
    ,  
    Liberati
    A
    .  et al. 
    GRADE Working Group
    Incorporating considerations of resources use into grading recommendations.
    BMJ
    2008
    336
    1170
    3
     PubMed
  22. Schünemann
    HJ
    ,  
    Schünemann
    AH
    ,  
    Oxman
    AD
    ,  
    Brozek
    J
    ,  
    Glasziou
    P
    ,  
    Jaeschke
    R
    .  et al. 
    GRADE Working Group
    Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
    BMJ
    2008
    336
    1106
    10
     PubMed
  23. Guyatt
    GH
    ,  
    Oxman
    AD
    ,  
    Kunz
    R
    ,  
    Falck-Ytter
    Y
    ,  
    Vist
    GE
    ,  
    Liberati
    A
    .  et al. 
    GRADE Working Group
    Going from evidence to recommendations.
    BMJ
    2008
    336
    1049
    51
     PubMed
  24. Guyatt
    GH
    ,  
    Oxman
    AD
    ,  
    Kunz
    R
    ,  
    Vist
    GE
    ,  
    Falck-Ytter
    Y
    ,  
    Schünemann
    HJ
    .  
    GRADE Working Group
    What is “quality of evidence” and why is it important to clinicians?
    BMJ
    2008
    336
    995
    8
     PubMed
  25. Guyatt
    GH
    ,  
    Oxman
    AD
    ,  
    Vist
    GE
    ,  
    Kunz
    R
    ,  
    Falck-Ytter
    Y
    ,  
    Alonso-Coello
    P
    .  et al. 
    GRADE Working Group
    GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
    BMJ
    2008
    336
    924
    6
     PubMed
  26. Sanders
    GD
    ,  
    Bayoumi
    AM
    ,  
    Sundaram
    V
    ,  
    Bilir
    SP
    ,  
    Neukermans
    CP
    ,  
    Rydzak
    CE
    .  et al. 
    Cost-effectiveness of screening for HIV in the era of highly active antiretroviral therapy.
    N Engl J Med
    2005
    352
    570
    85
     PubMed
  27. Paltiel
    AD
    ,  
    Weinstein
    MC
    ,  
    Kimmel
    AD
    ,  
    Seage
    GR
    3rd
    ,  
    Losina
    E
    ,  
    Zhang
    H
    .  et al. 
    Expanded screening for HIV in the United States—an analysis of cost-effectiveness.
    N Engl J Med
    2005
    352
    586
    95
     PubMed
  28. Paltiel
    AD
    ,  
    Walensky
    RP
    ,  
    Schackman
    BR
    ,  
    Seage
    GR
    3rd
    ,  
    Mercincavage
    LM
    ,  
    Weinstein
    MC
    .  et al. 
    Expanded HIV screening in the United States: effect on clinical outcomes, HIV transmission, and costs.
    Ann Intern Med
    2006
    145
    797
    806
     PubMed
  29. Sanders
    GD
    ,  
    Bayoumi
    AM
    ,  
    Holodniy
    M
    ,  
    Owens
    DK
    .  
    Cost-effectiveness of HIV screening in patients older than 55 years of age.
    Ann Intern Med
    2008
    148
    889
    903
     PubMed
  30. Walensky
    RP
    ,  
    Freedberg
    KA
    ,  
    Weinstein
    MC
    ,  
    Paltiel
    AD
    .  
    Cost-effectiveness of HIV testing and treatment in the United States.
    Clin Infect Dis
    2007
    45
    Suppl 4
    S248
    54
     PubMed
  31. Owens
    DK
    ,  
    Harris
    RA
    ,  
    Scott
    PM
    ,  
    Nease
    RF
    Jr
    .  
    Screening surgeons for HIV infection. A cost-effectiveness analysis.
    Ann Intern Med
    1995
    122
    641
    52
     PubMed
  32. Sanders
    GD
    ,  
    Hlatky
    MA
    ,  
    Owens
    DK
    .  
    Cost-effectiveness of implantable cardioverter-defibrillators.
    N Engl J Med
    2005
    353
    1471
    80
     PubMed
  33. Gage
    BF
    ,  
    Cardinalli
    AB
    ,  
    Albers
    GW
    ,  
    Owens
    DK
    .  
    Cost-effectiveness of warfarin and aspirin for prophylaxis of stroke in patients with nonvalvular atrial fibrillation.
    JAMA
    1995
    274
    1839
    45
     PubMed
  34. Eckman
    MH
    ,  
    Rosand
    J
    ,  
    Greenberg
    SM
    ,  
    Gage
    BF
    .  
    Cost-effectiveness of using pharmacogenetic information in warfarin dosing for patients with nonvalvular atrial fibrillation.
    Ann Intern Med
    2009
    150
    73
    83
     PubMed
  35. Brown
    AD
    ,  
    Garber
    AM
    .  
    Cost-effectiveness of 3 methods to enhance the sensitivity of Papanicolaou testing.
    JAMA
    1999
    281
    347
    53
     PubMed
  36. Freedberg
    KA
    ,  
    Losina
    E
    ,  
    Weinstein
    MC
    ,  
    Paltiel
    AD
    ,  
    Cohen
    CJ
    ,  
    Seage
    GR
    .  et al. 
    The cost effectiveness of combination antiretroviral therapy for HIV disease.
    N Engl J Med
    2001
    344
    824
    31
     PubMed
  37. Khazeni
    N
    ,  
    Hutton
    DW
    ,  
    Garber
    AM
    ,  
    Hupert
    N
    ,  
    Owens
    DK
    .  
    Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009.
    Ann Intern Med
    2009
    151
    829
    39
     PubMed
  38. The Center for the Evaluation of Value and Risk in Health. Cost-Effectiveness Analysis (CEA) Registry. Accessed athttps://research.tufts-nemc.org/cear/default.aspxon 22 September 2010.
  39. Detsky
    AS
    ,  
    Naglie
    IG
    .  
    A clinician's guide to cost-effectiveness analysis.
    Ann Intern Med
    1990
    113
    147
    54
     PubMed
  40. Bendavid
    E
    ,  
    Young
    SD
    ,  
    Katzenstein
    DA
    ,  
    Bayoumi
    AM
    ,  
    Sanders
    GD
    ,  
    Owens
    DK
    .  
    Cost-effectiveness of HIV monitoring strategies in resource-limited settings: a southern African analysis.
    Arch Intern Med
    2008
    168
    1910
    8
     PubMed
  41. Garber
    AM
    ,  
    Phelps
    CE
    .  
    Economic foundations of cost-effectiveness analysis.
    J Health Econ
    1997
    16
    1
    31
     PubMed
  42. National Institute for Health and Clinical Excellence. Guide to the Methods of Technology Appraisal. Published June 2008. Accessed atwww.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdfon 9 December 2010.
  43. Hutubessy
    R
    ,  
    Chisholm
    D
    ,  
    Edejer
    TT
    .  
    Generalized cost-effectiveness analysis for national-level priority-setting in the health sector.
    Cost Eff Resour Alloc
    2003
    1
    8
     PubMed
  44. Braithwaite
    RS
    ,  
    Meltzer
    DO
    ,  
    King
    JT
    Jr
    ,  
    Leslie
    D
    ,  
    Roberts
    MS
    .  
    What does the value of modern medicine say about the $50,000 per quality-adjusted life-year decision rule?
    Med Care
    2008
    46
    349
    56
     PubMed
  45. Ubel
    PA
    ,  
    Hirth
    RA
    ,  
    Chernew
    ME
    ,  
    Fendrick
    AM
    .  
    What is the price of life and why doesn't it increase at the rate of inflation?
    Arch Intern Med
    2003
    163
    1637
    41
     PubMed
  46. Weinstein
    MC
    .  
    How much are Americans willing to pay for a quality-adjusted life year? [Editorial].
    Med Care
    2008
    46
    343
    5
     PubMed
  47. National Institute for Health and Clinical Excellence. Cost saving guidance. Accessed atwww.nice.org.uk/usingguidance/benefitsofimplementation/costsavingguidance.jspon 22 September 2010.
  48. American College of Physicians
    Information on cost-effectiveness: an essential product of a national comparative effectiveness program.
    Ann Intern Med
    2008
    148
    956
    61
     PubMed
  49. Fuchs
    VR
    .  
    Eliminating “waste” in health care.
    JAMA
    2009
    302
    2481
    2
     PubMed

Appendix

Value

We use value to reflect an assessment of whether the benefits of an intervention justify its costs. Value is sometimes used in the decision-making and quality-of-life literature to refer to the importance that patients (or society) place on a health state.

Cost–Benefit Analysis

Cost–benefit analysis differs from cost-effectiveness analysis in that health outcomes are also expressed in dollars rather than in units of health. Thus, both the costs and the benefits of an intervention are expressed in dollars, and determining whether the benefits outweigh the costs is clear. Alternative methods are available for assessing health benefits in terms of dollars; however, because of the challenges inherent in assigning a monetary value to health outcomes, cost–benefit analysis has been used less frequently than cost-effectiveness analysis in health care settings.

Incremental Cost-Effectiveness Ratio

The incremental cost-effectiveness ratio is defined as the difference in costs divided by the difference in health benefit when 2 strategies are compared. For example, an analysis evaluating the cost-effectiveness of ICDs compared with medical therapy would use the following formula to determine the incremental cost-effectiveness ratio:
 
It is important to distinguish the incremental cost-effectiveness ratio from the average cost-effectiveness ratio; the latter is simply the cost of an intervention divided by the benefit. The average cost-effectiveness ratio can provide misleading estimates of the resources required to adopt a strategy and should not be used for decision making.

Dominance and Extended Dominance

The Figure demonstrates the concept of dominance. A dominated intervention is more expensive and less effective than another intervention. The Appendix Figure denotes another possible situation in which we show interventions A, B, C, and D as in the Figure, but we now also include intervention E, which is to the right and below the cost-effectiveness frontier.
Appendix Figure.
Extended dominance.

The cost-effectiveness ratios of interventions are shown on the lines between the interventions. Intervention E costs $35 000 per QALY gained relative to intervention A. Intervention B costs $15 000 per QALY gained relative to intervention E. Intervention D is dominated. Intervention E can be eliminated through extended dominance. QALY = quality-adjusted life-year.

In the Appendix Figure, intervention E is better than intervention A and more expensive but not as good or costly as intervention B. Unlike intervention D, which is dominated, no intervention is both better and less expensive than intervention E. In general, we would never choose intervention E; if our cost-effectiveness threshold were more than $35 000 per QALY gained, which would be necessary to choose intervention E over intervention A, we would always choose intervention B, which is better than intervention E and more cost effective ($25 000 per QALY gained rather than $35 000 per QALY gained). In this situation, we say that intervention E should not be chosen because we can eliminate it through extended dominance. In summary, we would not choose intervention D because intervention A is better and less expensive (strict dominance), and we would not choose intervention E because intervention B is better and has a more favorable cost-effectiveness ratio (extended dominance).
Figure.
Costs and benefits of 4 hypothetical interventions.

As the line between interventions becomes more horizontal, the cost-effectiveness ratio becomes less favorable because costs are increasing faster than benefits are. The slope of the line between 2 interventions represents the reciprocal of the cost-effectiveness ratio. A lower incremental cost-effectiveness ratio denotes more favorable cost-effectiveness. The lines between interventions A, B, and C are called the cost-effectiveness frontier. Any intervention with costs and QALYs below and to the right of the cost-effectiveness frontier would be dominated (such as intervention D), or less cost-effective than interventions on the frontier. QALY = quality-adjusted life-year.

Appendix Figure.
Extended dominance.

The cost-effectiveness ratios of interventions are shown on the lines between the interventions. Intervention E costs $35 000 per QALY gained relative to intervention A. Intervention B costs $15 000 per QALY gained relative to intervention E. Intervention D is dominated. Intervention E can be eliminated through extended dominance. QALY = quality-adjusted life-year.

Table 1.

Cost-Effectiveness Analysis of Implantable Cardioverter-Defibrillators

Table 1.

Table 2.

Cost, Benefit, and Value of Medical Interventions

Table 2.

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

In this video, Paul Shekelle, MD, PhD, offers additional insight into his clinical guideline, "High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions. (2:35)


High-Value, Cost-Conscious Health Care

This feature is available only to Registered Users

Subscribe/Learn More
Submit a Comment

0 Comments

PDF
Not Available
Citations
Citation

Owens DK, Qaseem A, Chou R, et al, for the Clinical Guidelines Committee of the American College of Physicians. High-Value, Cost-Conscious Health Care: Concepts for Clinicians to Evaluate the Benefits, Harms, and Costs of Medical Interventions. Ann Intern Med. 2011;154:174–180. doi: https://doi.org/10.7326/0003-4819-154-3-201102010-00007

Download citation file:

  • Ris (Zotero)
  • EndNote
  • BibTex
  • Medlars
  • ProCite
  • RefWorks
  • Reference Manager

© 2019

×
Permissions

Published: Ann Intern Med. 2011;154(3):174-180.

DOI: 10.7326/0003-4819-154-3-201102010-00007

185 Citations

See Also

Diagnostic Imaging for Low Back Pain: Advice for High-Value Health Care From the American College of Physicians
“Value for Money”: Use With Care
High-Value, Cost-Conscious Health Care
Screening, Monitoring, and Treatment of Stage 1 to 3 Chronic Kidney Disease
Screening, Monitoring, and Treatment of Stage 1 to 3 Chronic Kidney Disease
Screening, Monitoring, and Treatment of Stage 1 to 3 Chronic Kidney Disease
Screening, Monitoring, and Treatment of Stage 1 to 3 Chronic Kidney Disease
Screening, Monitoring, and Treatment of Stage 1 to 3 Chronic Kidney Disease
View MoreView Less

Related Articles

Triple Therapy Versus Biologic Therapy for Active Rheumatoid Arthritis: A Cost-Effectiveness Analysis
Annals of Internal Medicine; 167 (1): 8-16
Cost-Effectiveness of Screening for Asymptomatic Left Ventricular Dysfunction in Childhood Cancer Survivors
Annals of Internal Medicine; 160 (10): 731-732
Cost-Effectiveness of Different Types of Evaluations Before Sports Participation in Young Athletes
Annals of Internal Medicine; 152 (5): I-40
Cost-Effectiveness of Biologics as First-Line Treatment of Rheumatoid Arthritis: Case Closed?
Annals of Internal Medicine; 151 (9): 668-669
View MoreView Less

Journal Club

Review: Newer second-line drugs for diabetes are not more cost-effective than sulfonylureas
Annals of Internal Medicine; 168 (2): JC8
In treatment-resistant depression, adding cognitive–behavioral therapy to usual care was cost-effective at 1 y
Annals of Internal Medicine; 160 (12): JC13
Compared with warfarin, high-dose dabigatran might be cost-effective for preventing stroke in older patients with nonvalvular AF
Annals of Internal Medicine; 154 (12): JC6-12
Screening for abdominal aortic aneurysm (AAA) in men 65 to 74 years of age was cost-effective for AAA mortality at 10 years
Annals of Internal Medicine; 151 (10): JC5-15
View MoreView Less

Related Topics

Guidelines
High Value Care

Guidelines, High Value Care.

PubMed Articles

Neonatal encephalopathy therapy optimization for better neuroprotection with inhalation of CO2: the HENRIC feasibility and safety trial.
Pediatr Res 2019.
Development of a separation framework for effects-based targeted and non-targeted toxicological screening of water and wastewater.
Water Res 2019;
View More

Results provided by: PubMed

CME/MOC Activity Requires Users to be Registered and Logged In.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
The Comments Feature Requires Users to be Registered and Logged In.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
link to top

Content

  • Home
  • Latest
  • Issues
  • Channels
  • CME/MOC
  • In the Clinic
  • Journal Club
  • Web Exclusives

Information For

  • Author Info
  • Reviewers
  • Press
  • Readers
  • Institutions / Libraries / Agencies
  • Advertisers

Services

  • Subscribe
  • Renew
  • Alerts
  • Current Issue RSS
  • Latest RSS
  • In the Clinic RSS
  • Reprints & Permissions
  • Contact Us
  • Help
  • About Annals
  • About Mobile
  • Patient Information
  • Teaching Tools
  • Annals in the News
  • Share Your Feedback

Awards and Cover

  • Personae (Cover Photo)
  • Junior Investigator Awards
  • Poetry Prize

Other Resources

  • ACP Online
  • Career Connection
  • ACP Advocate Blog
  • ACP Journal Wise

Follow Annals On

  • Twitter Link
  • Facebook Link
acp link acp
silverchair link silverchair

Copyright © 2019 American College of Physicians. All Rights Reserved.

Print ISSN: 0003-4819 | Online ISSN: 1539-3704

Privacy Policy

|

Conditions of Use

This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy. | Accept
×

You need a subscription to this content to use this feature.

×
PDF Downloads Require Access to the Full Article.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
INSTITUTIONAL SIGN IN
Open Athens|Shibboleth|Log In
Annals of Internal Medicine
PURCHASE OPTIONS
Buy This Article|Subscribe
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
Access to this Free Content Requires Users to be Registered and Logged In. Please Choose One of the Following Options
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×