The Development of Clinical Practice Guidelines and Guidance Statements of the American College of Physicians: Summary of Methods FREE

The American College of Physicians (ACP) established its evidence-based clinical practice guidelines program in 1981. The ACP founded the Clinical Guidelines Committee (formerly known as the Clinical Efficacy Assessment Subcommittee [CEAS] and Clinical Efficacy Assessment Technical Advisory Committee [CEATAC]) with the charge to evaluate medical advances and develop clinical recommendations. Early ACP guidelines addressed diagnostic tests and technologies. Over the years, the clinical practice guideline program evolved, and current ACP clinical guidelines address screening, diagnosis, and treatment (www.acponline.org/clinical_information/guidelines) of diseases relevant to internal medicine and its subspecialties. Clinical recommendations from ACP represent the official opinion of the College.

The goal of the ACP is to provide clinicians with clinical guidelines based on the best available evidence; to make recommendations on the basis of that evidence; to inform clinicians of when there is no evidence; and finally, to help clinicians deliver the best health care possible. The program develops 2 types of clinical recommendations: clinical practice guidelines and clinical guidance statements. Clinical practice guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (1). The ACP guidelines are based on a systematic review of the literature. The ACP also produces clinical guidance statements, described in the “Guideline Review for Clinical Guidance Statements” section, which are based on a review of existing guidelines. Clinical recommendations may not apply to every patient or all clinical situations. The target audience for ACP clinical recommendations is all internists and other clinicians. The target patient population is persons with a clinical problem being addressed in a clinical recommendation.

The ACP's Clinical Guidelines Committee and the staff of the Clinical Programs and Quality of Care Department develop the clinical recommendations. Members of the Clinical Guidelines Committee are physicians trained in internal medicine and its subspecialties and include clinical experts and experts in evidence synthesis and guideline development. The Clinical Guidelines Committee is composed of a chair who is appointed by the ACP's Board of Regents (ACP's governing board) to serve for a 1-year term, which can be renewed 3 times for a maximum of 4 years. In addition, the Board of Regents appoints 11 other committee members to serve a 1-year term, which can be renewed annually for a maximum of 4 years. An immediate past chair serves a 1-year term after the expiration of his or her term as chair, with no reappointment (maximum of 1 year). The purpose of this position is to maintain continuity and to help completion of guidelines that are already in the process of development. The members of the Clinical Guidelines Committee, like all other College governance committees, must be ACP members in good standing. The nomination and appointment to Clinical Guidelines Committee follows the standard ACP procedures for selection of committee members by the governance committees of the College.

The Clinical Guidelines Committee meets in regularly scheduled face-to-face meetings and conference calls to review and assess the development of various guidelines and guidance statements.

At each meeting, all members of the ACP Board of Regents, Clinical Guidelines Committee, and ACP staff declare any potential financial and nonfinancial conflicts of interest that refer to relationships that a reasonable reader of a guideline would wish to know about and that if not disclosed could compromise the interpretation of the ACP guideline. Examples of financial conflicts of interests include ownership of stocks or shares, paid employment or consultancy, board membership, patent applications, research grants (from any source, restricted or unrestricted), travel grants and honoraria for speaking or participating at meetings, and gifts. Examples of nonfinancial conflicts of interests include leadership or close involvement in an advocacy group that stands to gain from a Clinical Guidelines Committee member's opinion; being a chair or member of another guideline committee relevant to the topic under discussion; acting as an expert witness or having a membership (in a government or other advisory board) or relationship (paid or unpaid) with organizations and funding bodies (including nongovernmental organizations, research institutions, or charities), or a membership in a lobbying or advocacy organization; writing or consulting for an educational company; having personal relationships (that is, a friend, spouse, family member, current or previous mentor, or adversary) with persons involved in the submission or evaluation of a paper, such as authors, reviewers, editors, or members of the editorial board of a Public Library of Science journal; and having personal convictions (political, religious, ideological, or other) related to a paper's topic that may interfere with an unbiased publication process (at the stage of authorship, peer review, editorial decision making, or publication).

The whole committee declares, discusses, and resolves any conflicts of interest of the Clinical Guidelines Committee members and ACP staff. If the conflicts of interest cannot be resolved, the member with the conflicts must recuse him- or herself from the discussion.

The ACP's clinical practice guidelines involve primary review of available evidence. The Clinical Guidelines Committee uses systematic literature reviews as the basis for guideline recommendations, as described in the “Evidence Review for Clinical Practice Guidelines” section. Clinical practice guidelines also identify gaps in evidence and direction for future research.

More recently, in response to the large number of existing and sometimes conflicting guidelines on topics of interests to clinicians, the Clinical Guidelines Committee has begun to develop clinical guidance statements. In contrast to clinical practice guidelines, which are based on a primary review of the available evidence, ACP clinical guidance statements involve review and critique of available guidelines.

The steps in the ACP guideline development process include selection of topics; determination of the scope of the topic; review of the evidence for clinical recommendations; and development, review, and approval of the recommendations. The ACP does not endorse guidelines that are developed by other organizations but does develop guidelines jointly with other professional societies.

Choosing a topic for a clinical practice guideline is the first step in the guideline development process. Candidate topics come from surveys of ACP members, other clinicians, the Clinical Guidelines Committee members, and other committees and governance of the ACP. In selecting a topic, the Clinical Guidelines Committee considers the following criteria: effect of the condition on morbidity and mortality, prevalence of the condition, whether effective health care is available, areas of uncertainty and evidence that current performance does not meet best practices, cost of the condition, relevance to internal medicine, and the likelihood that evidence is available to develop recommendations.

Current ACP guidelines address screening, diagnosis, and treatment of various diseases. The ACP guidelines usually focus on the effectiveness of interventions. However, when evidence about cost and cost-effectiveness of interventions is available and summarized in the systematic review, the Clinical Guidelines Committee incorporates this information into the guideline-development process.

The ACP generates an evidence-review paper for its clinical practice guidelines by following 1 of 3 pathways.

1. The ACP nominates the topics to the Agency for Healthcare Research and Quality (AHRQ) for an evidence review. If the AHRQ selects a topic, it commissions the topic to one of its evidence-based practice centers to complete an evidence review. The evidence reports produced by the evidence-based practice centers are the basis for many ACP guidelines. These comprehensive evidence reports are systematic literature reviews and are available to the public.

2. The ACP commissions evidence reports on interest out of its operational fund.

3. The ACP collaborates with other specialty societies on mutual topics of interests and jointly supports the development of the evidence reports that are commissioned.

The evidence-review paper serves as a companion piece to and foundation for the ACP's clinical practice guidelines. This paper summarizes evidence in evidence tables, analyzes the data, and synthesizes the available evidence.

The key questions and scope for the evidence-review papers are developed with input from the Clinical Guidelines Committee. The evidence-review paper is a comprehensive systematic review or meta-analysis that addresses the clinical topic area under review. The methods section specifies the criteria that are used to identify evidence related to each of the key questions for inclusion in the review. Quality of evidence is evaluated by using the ACP's Guideline Grading System, which is adopted from the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) system (Table 1). Evidence reviews for ACP guidelines provide information about whether the studies included in the evidence review are reliable and accurate and provide reasonable assessments of potential adverse events. Evidence reviews include information on systematic gaps in the literature, particularly with respect to areas of clinical importance and relevance.

Drafts of the evidence reviews are presented to the Clinical Guidelines Committee for review and comments. The reviews supported by the AHRQ also go through an additional external review process.

The evidence reviews also go through a statistical peer-review process by statisticians at Annals of Internal Medicine during its early stages of development.

A Clinical Guidelines Committee guideline subpanel for each clinical practice guideline is composed of the chair of the Clinical Guidelines Committee, 2 to 3 additional members of the Clinical Guidelines Committee, and 1 to 2 ACP staff members. The guideline subpanel members participate in various conference calls or in-person meetings, as needed, to discuss any issues and progress and provide feedback to the authors. Evidence-review authors discuss the timeline with the Clinical Guidelines Committee, which is usually within 1 year from the beginning of the project.

The ACP's clinical guidance statements involve review of available guidelines that other organizations developed. The ACP finances the development of its clinical guidance statements out of its operational funds.

We use a standardized instrument to evaluate relevant and available guidelines. Currently, we use the Appraisal of Guidelines Research and Evaluation instrument (Appendix Table), which asks 23 questions in 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence (2). Each guideline is scored in each domain. Among the criteria we consider an explicit link between evidence and recommendations, systematic search and selection methods, and whether methods for formulating recommendations are described.

A subpanel for each clinical guidance statement is similar in composition and activity to the clinical guideline subpanel.

The Clinical Guidelines Committee evaluates the evidence presented in the evidence reviews and uses them as the foundation for clinical recommendations. The recommendations are developed on the basis of evaluation and, where possible, quantification of the magnitude of benefits, harms, and costs; resource issues; implementation considerations; patient and caregiver concerns; and ethical and legal matters. Although subjective judgments are part of crafting recommendations, ACP recommendations are evidence-based to the extent possible. The Clinical Guidelines Committee is explicit about the scientific rationale for its recommendations. The Clinical Guidelines Committee uses an informal process to evaluate and formulate the recommendations on the basis of the evidence. The final clinical recommendations are approved by voting once there is a quorum of two thirds of the Clinical Guidelines Committee members present and approval by at least two thirds of the Clinical Guidelines Committee members present.

Patients' preferences are a key factor in many clinical decisions and are carefully considered during the development of ACP guidelines and guidance statement. By “patient preferences,” we mean a patient's view about the quality of life experienced with different states of health, including medical treatments (3). Formal methods for incorporating patient preferences into guidelines are available (4) but require decision analytic models, which are usually not available to the committee. Where available, we systematically search and review published literature on patient preference. Nevertheless, to the extent possible, we note clinical recommendations for which patient preferences can or should play an important role. For example, our guideline on mammography screening (5) highlights the critical importance of patient preferences in the decision to undergo screening mammography. We use the GRADE system, described below in the “Grading System for the Quality of Evidence and Strength of Recommendations” section, to denote situations in which patient preferences are an important consideration for clinical decisions.

After the Clinical Guidelines Committee reviews and approves a clinical guideline or guidance statement, these papers are then presented for final voting and approval as ACP policy to the ACP's Board of Regents, the highest body of ACP. Simultaneously, we invite the Board of Governors, which represents members from all 50 states and territories, as well as our international members, to provide input.

The ACP clinical practice guidelines, clinical guidance statements, and evidence reviews undergo an independent peer-review process at Annals of Internal Medicine before publication. On some occasions, ACP clinical practice guidelines and clinical guidance statements are also made available for external peer review in which outside reviewers and experts may be invited to comment on the clinical practice guidelines and clinical guidance statements before submission for publication.

The ACP's grading system for the quality of evidence and strength of recommendations is adapted from the classification developed by the GRADE workgroup (6). Although the GRADE system works best for treatment recommendations, it can be used to grade the quality of evidence and strength of recommendations for diagnostic tests or strategies (7). Table 1 summarizes ACP's guideline grading system, and Table 2 summarizes the interpretation of the evidence and its link to the recommendations.

Evidence is considered high quality when it is obtained from 1 or more well-designed and well-executed randomized, controlled trials (RCTs) that yield consistent and directly applicable results. This also means that further research is very unlikely to change our confidence in the estimate of effect.

Evidence is considered moderate quality when it is obtained from RCTs with important limitations—for example, biased assessment of the treatment effect, large loss to follow-up, lack of blinding, unexplained heterogeneity (even if it is generated from rigorous RCTs), indirect evidence originating from similar (but not identical) populations of interest, and RCTs with a very small number of participants or observed events. In addition, evidence from well-designed controlled trials without randomization, well-designed cohort or case–control analytic studies, and multiple time series with or without intervention are in this category. Moderate-quality evidence also means that further research will probably have an important effect on our confidence in the estimate of effect and may change the estimate.

Evidence obtained from observational studies would typically be rated as low quality because of the risk for bias. Low-quality evidence means that further research is very likely to have an important effect on our confidence in the estimate of effect and will probably change the estimate. However, the quality of evidence may be rated as moderate or even high, depending on circumstances under which evidence is obtained from observational studies. Factors that may contribute to upgrading the quality of evidence include a large magnitude of the observed effect, a dose–response association, or the presence of an observed effect when all plausible confounders would decrease the observed effect (8).

When the evidence is insufficient to determine for or against routinely providing a service, we grade the recommendation as “insufficient evidence to determine net benefits or risks.” Evidence may be conflicting, of poor quality, or lacking, and hence the balance of benefits and harms cannot be determined. Any estimate of effect that is very uncertain as evidence is either unavailable or does not permit a conclusion.

A strong recommendation means that benefits clearly outweigh risks and burden, or risks and burden clearly outweigh benefits.

When benefits are finely balanced with risks and burden or appreciable uncertainty exists about the magnitude of benefits and risks, a recommendation is classified as weak. Patient preferences may strongly influence the appropriate therapy.

The process for updating ACP guidelines is evolving. Because guidelines on different topics will probably go out of date at different rates (9 - 10), ACP plans an active surveillance program that will combine focused literature searches and expert judgment to assess whether a guideline is likely to be out of date. Guidelines identified as such may be updated in whole or in part or withdrawn if no update can be done.

All ACP clinical practice guidelines and clinical guidance statements are considered automatically withdrawn or invalid 5 years after publication or once an update has been issued.

Financial support for the development of ACP practice guidelines and guidance statements comes from the ACP operating budget. Financial support for joint guidelines may include support from the partner organizations and is disclosed in each specific joint guideline. Evidence reviews may be supported by the ACP operating budget or other external sources (such as the AHRQ) and any such support is disclosed.

Members of the Clinical Guidelines Committee are volunteers and do not receive any stipends except for reimbursement for travel-related costs, which comes out of the ACP operational budget.

All ACP clinical recommendations are considered public documents and are freely available at www.acponline.org/clinical_information/guidelines. In addition, most clinical recommendations are published in Annals of Internal Medicine, and reprints are available to all interested parties on request.