Skip Navigation
American College of Physicians Logo
  • Subscribe
  • Submit a Manuscript
  • Sign In
    Sign in below to access your subscription for full content
    INDIVIDUAL SIGN IN
    Sign In|Set Up Account
    You will be directed to acponline.org to register and create your Annals account
    INSTITUTIONAL SIGN IN
    Open Athens|Shibboleth|Log In
    Annals of Internal Medicine
    SUBSCRIBE
    Subscribe to Annals of Internal Medicine.
    You will be directed to acponline.org to complete your purchase.
Annals of Internal Medicine Logo Menu
  • Latest
  • Issues
  • Channels
  • CME/MOC
  • In the Clinic
  • Journal Club
  • Web Exclusives
  • Author Info
Advanced Search
  • ‹ PREV ARTICLE
  • This Issue
  • NEXT ARTICLE ›
Medicine and Public Policy |4 August 2009

Comparative Effectiveness Research: A Report From the Institute of Medicine Free

Harold C. Sox, MD, Editor; Sheldon Greenfield, MD

Harold C. Sox, MD, Editor

Sheldon Greenfield, MD

Article, Author, and Disclosure Information
Author, Article, and Disclosure Information
Potential Financial Conflicts of Interest: None disclosed.
Corresponding Author: Harold C. Sox, MD, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.
Current Author Addresses: Dr. Sox: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.
Dr. Greenfield: Center for Health Policy Research, University of California, Irvine, 111 Academy, Suite 220, Irvine, CA 92697.
  • From the American College of Physicians, Philadelphia, Pennsylvania, and University of California, Irvine, Irvine, California.
×
  • ‹ PREV ARTICLE
  • This Issue
  • NEXT ARTICLE ›
Jump To
  • Full Article
  • FULL ARTICLE
  • FULL ARTICLE
    • Definition of CER
    • High-Priority Research Topics
    • Establishing a Sustainable National Program of CER
    • Personal Reflections
      1. References
    • Appendix: Recommendations to Establish the Infrastructure for a Successful CER Initiative
  • Figures
  • Tables
  • Supplements
  • Audio/Video
  • Summary for Patients
  • Clinical Slide Sets
  • CME / MOC
  • Comments
  • Twitter Link
  • Facebook Link
  • Email Link
More
  • LinkedIn Link
  • CiteULike Link
Join the dialogue on health care reform. Comment on the perspectives published in Annals and offer ideas of your own. All thoughtful voices should be heard.
On 30 June 2009, another milestone was achieved in progress toward a national system to promote medical research that focuses on decision making by physicians and patients. The first steps occurred in late 2007 and early 2008 with a seminal article (1) and an Institute of Medicine (IOM) report that called for a national initiative of research that would support better decision making about interventions in health care (2). One of us was a coeditor of the IOM report. A third milestone was reached when both presidential candidates endorsed this concept. A fourth milestone came when the president signed into law the American Recovery and Reinvestment Act of 2009 (ARRA); this act allotted $1.1 billion to support this form of research, which had become known as comparative effectiveness research (CER). The legislation created a federal council on CER and asked the IOM to elicit input from a broad array of stakeholders on which research topics should have the highest-priority for funding through the ARRA and to then develop a list of the highest-priority topics for the Secretary of Health and Human Services to consider.
By law, the Federal Coordinating Council and the IOM committee to set priorities for comparative effectiveness research reported to the Secretary on 30 June 2009. This issue contains both this article, which is a commentary on the IOM committee report (3) by the committee's cochairs, and a perspective on better research methods for CER (4).

Definition of CER

The IOM committee quickly settled on a working definition of CER, which consisted of the elements of earlier definitions reduced to 2 sentences:

CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition, or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.

Two key elements that are embedded in this definition are the direct comparison of effective interventions and their study in patients who are typical of day-to-day clinical care. These features would ensure that the research would provide information that decision makers need to know, as would a third feature, research designed to identify the clinical characteristics that predict which intervention would be most successful in an individual patient. The same research design would also help policymakers by identifying subpopulations of patients that are more likely to benefit from one intervention than the other.

High-Priority Research Topics

The IOM committee sought advice from a broad range of stakeholders and received it in 3 forms: approximately 90 letters; 54 oral presentations at a day-long hearing in Washington, DC; and a Web-based nomination form. Overall, the committee received more than 2606 nominations from 1758 individual responders within 3 weeks. In a 3-step voting process, the committee identified 100 high-priority topics, which it ranked in quartiles. At each stage, the committee sought to have a balanced portfolio of topics. Table 5-1 in the report (available at http://www.iom.edu/cerpriorities) lists the 100 highest-ranked topics.

Establishing a Sustainable National Program of CER

The U.S. Congress is currently debating legislation that would guarantee health insurance to more Americans while reforming the U.S. health system. A key aim of the legislation is to moderate the rate at which the cost of health care increases while improving the quality of care. Many leaders believe that reaching this goal will require a sustained effort to produce better evidence to inform decision making. Government officials have characterized the $1.1 billion allocated to CER as a down payment on a national program of CER. As of this writing, the House of Representatives bill proposes a national program of CER, and the Senate Finance Committee white paper does likewise. Accordingly, the IOM committee made several recommendations aimed at a sustainable, trustworthy national CER initiative. The 8 provides the full text of these recommendations.

Governance

The 2008 IOM committee recommended establishing a national program “with authority, overarching responsibility, sustained resources, and adequate capacity to ensure production of credible, unbiased information about what is known and what is not known about clinical effectiveness” (2). The current committee expanded this recommendation by calling upon the secretary of the Department of Health and Human Services “to establish a mechanism—such as a coordinating advisory body—with the authority to strategize, organize, monitor, evaluate, and report on the implementation and impact of the CER Program.” The committee recognized that many agencies of the federal government are engaged in research that fits some or all of the key elements of CER. To make the best use of the funds allocated for CER, these agencies should be jointly accountable for spending CER funds on research that reflects the same conceptualization of CER. Therefore, these disparate efforts should use a single definition of CER and the same definitions of outcomes and measures of function and illness. This form of cross-agency coordination is compatible with each agency focusing on the form of research with which it has the greatest experience.

Active Involvement of Consumers in CER

The 2009 ARRA created a means for coordinating federal CER efforts: the Federal Coordinating Council. Seeking to complement the functions of the Council, the IOM committee focused on the potential contribution of consumers of health care to CER and recommended involving them at all levels of the national CER initiative. The recommendation is as follows: “The CER Program should fully involve consumers, patients and their caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.”
The concept that animates this recommendation is a radical but logical departure from past practice: When a principal aim of research is to inform decision makers, listen to the decision makers. Meaningful participation will require consumers to learn more about research, just as they will teach researchers about how to frame their investigations to satisfy the needs of consumers who are also decision makers. This approach has precedent. For example, the National Institute for Health and Clinical Effectiveness in the United Kingdom teaches decision making to consumers who participate in the institute's program.

Methodology of CER

Remembering that the purpose of CER is to inform decision making, consider the principal forms of clinical research. Systematic reviews of the literature summarize a body of evidence. Over the past 2 decades, they have become indispensable to expert panels that formulate practice guidelines or policymakers who make insurance coverage decisions. These “macro” decision makers are starting to use decision models for the same purposes (5). Two broad categories of research are the starting point for systematic reviews: observational research and randomized trials. Large established databases provide an opportunity to link current health care practices to the outcome of care. These databases are representative of critical groups often omitted from randomized trials, but they contain limited data; of note, they seldom specify the rationale for medical decisions. These observational research methods have many advantages—speed, real-world decisions, large numbers of decisions and outcomes, and low cost—but they cannot escape a key limitation: Characteristics of the patient that drive real-life clinical decisions may also influence clinical outcomes, leading to uncertainty about whether they or the intervention itself causes the outcomes. Overcoming the limitations of observational research is the most important frontier of research on study methods.
To overcome these limitations, researchers often randomly assign patients to different interventions. This simple action eliminates much of the uncertainty that plagues the interpretation of observational research. However, we have too often used the conceptual elegance of randomization to answer the wrong questions. We ask, “Does this work?” when our readers want to know, “Is this better than that?” Thus, too often, randomized trials are so-called efficacy studies: They are designed to create near-ideal circumstances to see whether the intervention can possibly work. To this end, researchers exclude many types of patients from efficacy trials and often compare the intervention with a placebo. In these respects, efficacy trials are unlike the conditions that physicians and patients face in daily practice. Another article in this issue (4) describes several exciting new methods for making randomized trials more suitable for CER.
The examples of observational research and new randomized trial methods are the basis for the IOM committee's strong recommendation that a national CER program should support research in the methods of clinical research.

The Infrastructure for Observational Research

The committee believes that very large collections of the electronic records of patients can be a valuable resource for CER. Using these data sets, it is possible to compare the outcomes of using several effective interventions in a population that is representative of daily care, 2 features that are aligned with the goals of CER. The great numbers of patients in these data sets also makes it possible to study subgroups with precision and perhaps identify key predictors of response to an intervention, both of which would facilitate decision making at the individual and population level. However, using these data sets will require researchers to surmount several technical barriers, to say nothing of the analytic challenges of making crisp inferences from observational data.
The committee's recommendation is worded as follows: “The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to yield CER findings.”
The committee recognized the need to protect patients' privacy; it therefore called for development of methods for linking patient-level data from multiple sources. It also recognized that participating in a data network requires institutions to invest in the technical requirements and make a commitment to provide high-quality data. Accordingly, the national CER program should create incentives to participate and rewards for good performance.

The CER Workforce

Depending on the scale of the public's investment in CER, the workforce for CER may prove to be inadequate. The committee recommended that a national CER program invest in developing a research workforce with the requisite skills for CER. The national CER program should develop a strategic plan for meeting the CER workforce needs and invest in career development support at every level from graduate school to mid-career.

Translating Research Findings into Patient Care

The main reason for public interest in CER is the belief that better information will result in better health outcomes and more effective use of resources. Comparative effectiveness research is an important cog in a larger mechanism for promoting better health care practices. The committee believes that a national CER program will be perceived as successful if it funds research that improves the uptake of new knowledge and its translation into better decision making. Indeed, many of the high-ranked research topics addressed questions about the best methods to translate research results into action. In the end, whether CER will realize its potential will depend on other features of health care reform legislation, the profession's ability to mobilize the best in its members, and the ability of individual patients to engage their physicians in a dialogue about their own care.

Personal Reflections

Research that informs clinical decisions is everywhere, yet a national research initiative to improve decision making by patients and their physicians is a novel concept. We should all hope that the program goes well beyond the goal of providing better information for decision making. That mission is important. Indeed, it is the principal motivation for a national CER program. However, we believe that it is not enough. Somehow, and in concert with national professional organizations and local opinion leaders, a national program should promote the professional ethos that places the interests of patients and the larger community above all other considerations (6). Together, a national initiative that engages the medical profession and patients can justify a large public investment in CER.

References

  1. Wilensky
    GR
    .  
    Developing a center for comparative effectiveness information.
    Health Aff (Millwood)
    2006
    25
    w572
    85
    PubMed
    CrossRef
  2. Eden J, Wheatley B, McNeil B, Sox H, eds. Institute of Medicine. Knowing What Works in Health Care: A Roadmap for the Nation. Washington, DC: National Academies Pr; 2008.
  3. Ratner R, Eden J, Wolman D, Greenfield S, Sox H, eds. Institute of Medicine. Initial national priorities for comparative effectiveness research. Washington, DC: National Academies Pr; 2009.
  4. Luce
    BR
    ,  
    Kramer
    JM
    ,  
    Goodman
    SN
    ,  
    Connor
    JT
    ,  
    Tunis
    S
    ,  
    Whicher
    D
    .  et al. 
    Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change.
    Ann Intern Med
    2009
    151
    206
    9
  5. Zauber
    AG
    ,  
    Lansdorp-Vogelaar
    I
    ,  
    Knudsen
    AB
    ,  
    Wilschut
    J
    ,  
    van Ballegooijen
    M
    ,  
    Kuntz
    KM
    .  
    Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force.
    Ann Intern Med
    2008
    149
    659
    69
    PubMed
  6. Gawande A. The cost conundrum. The New Yorker. 2009;1 June. Accessed athttp://www.newyorker.com/reporting/2009/06/01/090601fa_fact_gawandeon 23 June 2009.

Appendix: Recommendations to Establish the Infrastructure for a Successful CER Initiative

Recommendation 5: The HHS Secretary should establish a mechanism—such as a coordinating advisory body—with the authority to strategize, organize, monitor, evaluate, and report on the implementation and impact of the CER Program.
Recommendation 6: The CER Program should fully involve consumers, patients and their caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.
  • The CER Program should develop strategies to reach out to, engage, support, educate, and, as necessary, prepare consumers, patients and their caregivers for leadership roles in these activities.

  • The CER Program should also encourage broad participation in CER research in order to create a representative evidence base that could help identify health disparities and inform decisions by patients in special population groups.

Recommendation 7: The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practical, and efficient clinical trials.
Recommendation 8: The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to yield CER findings.
  • The CER Program should ensure that CER researchers and institutions implement privacy and security protections in all their activities.

  • The CER Program should support the development of methodologies for linking patient-level data from multiple sources, and for analyzing the resulting data for CER.

  • The CER Program should promote widespread participation and provide incentives to data holders to participate in CER.

Recommendation 9: The CER Program should develop and support the workforce for CER to ensure the nation's capacity to carry out the CER research mission. Important next steps include:
  • Development of a strategic plan for research workforce development.

  • Long-term, sufficient funding for career development including expanding grants for graduate and postgraduate training opportunities in comparative effectiveness methods as well as career development grants and mid-career merit awards.

Recommendation 10: The CER Program should promote rapid adoption of CER findings and conduct research to identify the most effective strategies for disseminating new and existing CER findings to health care professionals, consumers, patients, and their caregivers and for helping them to implement these results in daily clinical practice.

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

This feature is available only to Registered Users

Subscribe/Learn More
Submit a Comment

0 Comments

PDF
Not Available
Citations
Citation

Sox HC, Greenfield S. Comparative Effectiveness Research: A Report From the Institute of Medicine. Ann Intern Med. ;151:203–205. doi: 10.7326/0003-4819-151-3-200908040-00125

Download citation file:

  • Ris (Zotero)
  • EndNote
  • BibTex
  • Medlars
  • ProCite
  • RefWorks
  • Reference Manager

© 2019

×
Permissions

Published: Ann Intern Med. 2009;151(3):203-205.

DOI: 10.7326/0003-4819-151-3-200908040-00125

327 Citations

See Also

Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change
View MoreView Less

Related Articles

Design of a National Distributed Health Data Network
Annals of Internal Medicine; 151 (5): 341-344
Oversight of Patient-Centered Outcomes Research: Recommendations From a Delphi Panel
Annals of Internal Medicine; 169 (8): 559-563
Comparative Effectiveness of Implementation Strategies for Blood Pressure Control in Hypertensive Patients: A Systematic Review and Meta-analysis
Annals of Internal Medicine; 168 (2): 110-120
Annals for Educators - 16 January 2018
Annals of Internal Medicine; 168 (2): ED2
View MoreView Less

Journal Club

In women with stress urinary incontinence, initial surgery was more effective than physiotherapy
Annals of Internal Medicine; 160 (4): JC7
What kind of randomized trials do patients and clinicians need?
Annals of Internal Medicine; 150 (10): JC5-2
View MoreView Less

Related Topics

Healthcare Delivery and Policy
High Value Care

Healthcare Delivery and Policy, High Value Care.

PubMed Articles

The Cost-Effectiveness of Financial Incentives for Viral Suppression: HPTN 065 Study.
Value Health 2019;22(2):194-202.
Radiology Research Funding: Current State and Future Opportunities.
Acad Radiol 2018;25(1):26-39.
View More

Results provided by: PubMed

CME/MOC Activity Requires Users to be Registered and Logged In.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
The Comments Feature Requires Users to be Registered and Logged In.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
link to top

Content

  • Home
  • Latest
  • Issues
  • Channels
  • CME/MOC
  • In the Clinic
  • Journal Club
  • Web Exclusives

Information For

  • Author Info
  • Reviewers
  • Press
  • Readers
  • Institutions / Libraries / Agencies
  • Advertisers

Services

  • Subscribe
  • Renew
  • Alerts
  • Current Issue RSS
  • Latest RSS
  • In the Clinic RSS
  • Reprints & Permissions
  • Contact Us
  • Help
  • About Annals
  • About Mobile
  • Patient Information
  • Teaching Tools
  • Annals in the News
  • Share Your Feedback

Awards and Cover

  • Personae (Cover Photo)
  • Junior Investigator Awards
  • Poetry Prize

Other Resources

  • ACP Online
  • Career Connection
  • ACP Advocate Blog
  • ACP Journal Wise

Follow Annals On

  • Twitter Link
  • Facebook Link
acp link acp
silverchair link silverchair

Copyright © 2019 American College of Physicians. All Rights Reserved.

Print ISSN: 0003-4819 | Online ISSN: 1539-3704

Privacy Policy

|

Conditions of Use

This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy. | Accept
×

You need a subscription to this content to use this feature.

×
PDF Downloads Require Access to the Full Article.
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
INSTITUTIONAL SIGN IN
Open Athens|Shibboleth|Log In
Annals of Internal Medicine
PURCHASE OPTIONS
Buy This Article|Subscribe
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
Access to this Free Content Requires Users to be Registered and Logged In. Please Choose One of the Following Options
Sign in below to access your subscription for full content
INDIVIDUAL SIGN IN
Sign In|Set Up Account
You will be directed to acponline.org to register and create your Annals account
Annals of Internal Medicine
CREATE YOUR FREE ACCOUNT
Create Your Free Account|Why?
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×