Comparative Effectiveness Research: A Progress Report FREE

Sixteen months ago, comparative effectiveness research (CER) began its rapid rise, when The American Recovery and Reinvestment Act of 2009 (referred to as “the Reinvestment Act” hereinafter) allocated $1.1 billion for CER. When President Obama signed the bill, the recipients of the funds (the National Institutes of Health [NIH], Agency for Health Research and Quality [AHRQ], and Office of the Secretary of the U.S. Department of Health and Human Services [OS-DHHS]) hurried to issue requests for proposals to develop CER infrastructure and to conduct CER studies. The Reinvestment Act also mandated an Institute of Medicine (IOM) study to establish national priorities for CER. Then, early in 2010, the health reform legislation established an ongoing national program in CER: the Patient-Centered Outcomes Research Institute (PCORI). This progress report summarizes how the NIH, AHRQ, and OS-DHHS are spending the $1.1 billion, how the IOM priority topics have fared in the agencies' funding programs, and the developing plans for a national CER program.

The health reform legislation (1), the IOM priority-setting committee (2), and the Federal Coordinating Council for CER (3) define “CER” similarly. The IOM report states that CER is “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policymakers to make informed decisions that will improve health care at both the individual and population levels” (2).

According to this definition, the purpose of CER is to help stakeholders, including patients and their physicians, to make informed choices among alternatives. This statement of purpose embraces the concept of individualized decision making, in which the clinical characteristics and preferences of the patient help to determine the choice among alternatives. Consequently, comparative studies will search for evidence that the response to treatment varies within the study population, a phenomenon called “treatment response heterogeneity.” How often it occurs is not known, but 2 randomized trials show that it can occur. In each trial, the treatment that was the most effective overall caused worse outcomes in 1 or more large subgroups (4 - 5).

Comparative effectiveness research has captured the attention of the stakeholders in health care. The sheer magnitude of the investment—$1.1 billion—would be enough to get researchers' attention, but so has the opportunity to address hitherto unexplored topics in clinical decision making. Payers hope that better decision making will reduce unnecessary expenditures, an urgent problem for the United States and many other nations. Because medical knowledge is often readily transferable, the U.S. government's commitment to fund CER is a global windfall. Leaders of health care delivery systems hope that CER results will help them to fully realize the benefits of investing in interoperable electronic medical records (6) and programs to improve quality. The U.S. public is the central stakeholder in CER. Some patients and their advocates worry that better evidence about an intervention may lead to denial of insurance coverage, Many others welcome the potential for better information to help them decide which test or treatment is best for them. Of note, all stakeholders are invited to play an active role in every aspect of CER, including priority setting, study design, and peer review. Stakeholders play an important role in the Effective Health Care Program of AHRQ, and they will play a similar role in the newly created PCORI.

Since 2003, AHRQ has had a congressional mandate to support CER. Section 1013 of the Medicare Modernization Act of 2003 authorized AHRQ to conduct research to compare the effectiveness of different tests and treatments. The AHRQ supports the Effective Health Care Program, a 3-part program that is built around CER and analyses to identify knowledge gaps, generation of new knowledge, and dissemination and translation. This program includes the Evidence-based Practice Centers, which perform systematic reviews to support public and private policymaking. It also includes the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) network of centers, whose main task is informing policymaking by responding to requests for short-term, fast-turnaround analysis of existing data. The AHRQ will support the DEcIDE network with Reinvestment Act funds. The agency has recently renewed the DEcIDE network for 3 more years. The Effective Health Care program also supports the Centers for Education and Research in Therapeutics program, which does research and educates the public about drugs, biologics, and devices; grants individual investigator awards; and supports institutional and individual training grants.

The AHRQ will spend its $300 million share of the Reinvestment Act funds in the following ways (7 - 8). First, it will support an ongoing process to scan the horizon for new and emerging issues for CER to address. Second, it will support 8 Evidence-based Practice Centers to continue the work of evidence synthesis and to identify evidence gaps that will inform the AHRQ planning process. Third, it will support projects that develop and test strategies to incorporate CER findings into clinical practice. Fourth, Clinical and Health Outcomes Initiative in Comparative Effectiveness (CHOICE) grants will be the centerpiece of the AHRQ effort to support the generation of evidence about comparative clinical effectiveness. Each of these ten $10 million grants will fund a team to conduct large, pragmatic studies that focus on at least 1 of the 14 AHRQ priority conditions (9). Fifth, AHRQ will invest heavily in registries to collect clinical information on patients with specific diseases or who receive specific services. Sixth, to meet the demand for an expanded CER workforce, AHRQ will support career development. Finally, AHRQ will invest funds to engage citizens in the CER enterprise.

The NIH received $400 million of the $1.1 billion CER allocation in the Reinvestment Act, a relatively small part of the $10.4 billion that the Reinvestment Act allocated to the NIH (10). The NIH committed most of the $400 million to specific projects by the end of September 2009, mostly by means of new funding mechanisms (challenge grants and “grand opportunity” grants) that were created in response to the Reinvestment Act. The agency acted quickly on these applications: The time from grant submission to funding the NIH CER projects was typically 5 to 6 months. Currently, the NIH supports 165 projects with Reinvestment Act funds (11), most of which directly address clinical questions. These questions range from the comparison of depression interventions after acute coronary syndromes to the comparative effectiveness of Web-based mobile support for tobacco cessation.

The OS-DHHS, which received $400 million in Reinvestment Act funds, focused its efforts on strengthening the infrastructure to support CER (12). The Secretary asked DHHS agencies to nominate topics; most of the selected topics had been proposed by AHRQ or by OS-DHHS, but some were generated by the NIH, the U.S. Food and Drug Administration Health Research and Services Administration, and the Centers for Medicare & Medicaid Services. Selected projects fall in 1 of 4 categories. Data infrastructure projects address the development of longitudinal claims databases, distributed data networks, and patient registries. Dissemination, translation, and implementation projects will focus on the translation of CER evidence to clinical settings. Research projects will identify the relative strengths and weaknesses of various medical interventions. Inventory and evaluation projects will catalogue CER activities and infrastructure in order to track national investments in CER going forward. In addition, much of the funding will support efforts to provide access to representative populations for observational research on community-based care, an important focus of CER. The Appendix Table describes these projects in more detail.

Although it is far too early to evaluate the outcome of the federal response to the Reinvestment Act funding, the process was impressive. The agencies had a legislative mandate to disburse the funds quickly. Given the ambitious deadline for obligating the funds, some people were concerned that the agencies would lose sight of the definition of CER in their haste to award funds quickly. These concerns proved groundless, as the funded projects do seem to address the purposes of CER. Another concern was whether the awards would reflect a consensus about national priorities for CER. The IOM priority-setting committee made its report on 30 June 2009, 4 months after the Reinvestment Act became law. The NIH and AHRQ were well advanced in their plans for disbursing Reinvestment Act funds by that time, but both agencies cross-checked their choices against the IOM list of 100 priority research questions, as did the OS-DHHS. The NIH determined that their choices included 88 of the 100 IOM priorities and subsequently issued requests for applications to cover the remaining topics (10). The AHRQ has issued several requests for applications that either asked applicants to address a specific IOM high-priority research question or required the applicant to propose a topic that addressed one of the IOM priorities.

The Reinvestment Act funding seems to have been a down payment on an ongoing national CER program. The Patient Protection and Affordable Care Act (PPACA) creates and funds PCORI, a public–private partnership to support CER. Starting in 2013, Medicare and every private health insurance company will pay a tax on each of their insured lives. Revenues from this tax will be held in a trust fund and used to support PCORI. The PCORI funding will therefore be exempt from the congressional budget process. Annual PCORI funding is estimated to be $500 million by 2014 (10,13). The PCORI will have a board of governors, whose members will be chosen by the Comptroller General by September 2010.

Currently, the only source of information about how PCORI will function is Section 6301 of the PPACA (1). The Table shows some of the functions of PCORI, and these suggest that the principal focus of PCORI will be to formulate a portfolio of research projects. Colleagues and I recently proposed a development process for research project portfolios that would strive to balance the conflicting requirements of broad coverage of high-priority topics, precisely interpretable research results, and a fixed research budget (14). To solve this optimization problem, we suggested that PCORI should specify both the research question and the most appropriate method to address it. The recommended study design would take into account the costs of alternative research designs, the tolerance for uncertainty about the validity of the results, and the importance of other research questions on the project agenda. To make these difficult assessments, PCORI would need intensive methodological and topic-specific advice. Section 6301 of the PPACA addresses this need by specifying a standing methodology committee and ad hoc expert advisory committees to address individual topics. According to the legislative language, PCORI will contract with federal agencies and private sector organizations to manage the funding and conduct of the research, which suggests that PCORI will outsource the entire process of soliciting research proposals, reviewing applications, making awards, and monitoring adherence to contractual obligations.

The current approach to funding much of CER contrasts sharply with traditional investigator-initiated research, in which the researcher proposes the topic and the methods of investigation. Funding agencies frequently propose the topic of CER, drawing upon agency priorities or the IOM list of 100 topics (2). Sometimes, the agency will also specify key elements of the study design, such as prospective (for example, a trial or a registry) or retrospective (for example, analysis of claim data or medical records). Specifying the research design is a deeper incursion on the autonomy of the researcher, but one that may be reasonable to address high-priority CER topics, which usually require enrolling large numbers of representative patients. A rationale for taking this step is the deep and widely shared knowledge of the small number of research designs for studying large populations. Therefore, ingenuity in choosing a research design is less important than understanding the clinical content of the research problem and experience in conducting high-quality, large-scale studies. Perhaps drawing a bright line between the design of a study and its execution is appropriate for taxpayer-funded large-scale clinical research that addresses high-priority questions. In the end, the conduct of clinical research is still the main challenge.

President Obama sees ever increasing health care costs as a principal threat to the nation's fiscal solvency and believes that research can help to solve the problem. Because he has taken the first steps to test this hypothesis, the immediate future of CER looks bright. In the past 16 months, federal legislation has launched CER, and federal agencies have spent a large bolus of CER funding expeditiously and wisely. However, compared with what lies ahead, these remarkable accomplishments will seem relatively trivial. In the next decade, the United States must absorb 32 million currently uninsured people into the health care system while simultaneously improving the quality of care and slowing cost increases. These accomplishments will require a transformation of U.S. health care. Comparative effectiveness research is a necessary part of this change, but it is hardly sufficient to achieve it. Even if it succeeds in providing clearer guidance on what works best and in whom, CER could still become the scapegoat for failure to achieve the larger goals of health care reform.