A Summary of the Methods That the National Clinical Guideline Centre Uses to Produce Clinical Guidelines for the National Institute for Health and Clinical Excellence FREE

The National Institute for Health and Clinical Excellence (NICE) produces clinical guidelines and guidance on public health and health technologies for the National Health Service in England, Wales, and Northern Ireland. The Scottish Intercollegiate Guidelines Network develops separate guidelines for Scotland. This article describes the process used to develop NICE clinical guidelines.

The National Institute for Health and Clinical Excellence commissions 4 external centers and 1 internal center to produce clinical guidelines on its behalf. The National Clinical Guideline Centre is the largest of these centers; it was established in 2009 from a merger of 4 smaller guideline-producing centers specializing in acute care, chronic conditions, primary care, and nursing and supportive care. The other 3 external centers that currently produce guidance are the National Collaborating Centre for Cancer, the National Collaborating Centre for Women's and Children's Health, and the National Collaborating Centre for Mental Health. Short clinical guidelines are produced by the internal center at NICE.

The National Institute for Health and Clinical Excellence has been publishing clinical guidelines since 2001, and more than 100 guidelines have been completed. The methods used to produce the guidelines have evolved but remain fundamentally focused on providing clear recommendations that are based on the best available evidence for the appropriate treatment and care of people with specific diseases and conditions. This is based on review of the clinical and cost-effectiveness of treatments and, for some guidelines, diagnostic and prognostic evidence, to support clinician decision making. In addition, several published and “in development” guidelines have covered service delivery and more generic issues, such as adherence to medication regimens and patient experience.

For the interest of an international audience, we summarize the methods that the National Clinical Guideline Centre uses to develop clinical guidelines for NICE. A full description of these methods is available in The Guidelines Manual 2009(1). For guidelines for which the scoping process started before 5 January 2009, the 2007 version of the guidelines manual applies (2). The AGREE instrument (3) is used to assess the quality of clinical guideline development process and reporting; we use the framework in AGREE to structure this article. The Table shows an overview of the guideline development process and timelines.

Suggestions for guideline topics come from diverse sources, including health care professionals, patients, carers, the general public, and the Department of Health or the Welsh Assembly Government. National panels then formally prioritize these topics, which are administered by NICE, and ministers at the Department of Health sign off on which topics should have guidelines commissioned. Selection of topics is based on many factors, including the burden of disease, the impact on resources, the importance of policy, whether there is inappropriate variation in practice across the country, and factors that affect the timeliness of or urgency for guidance to be produced. The topic selection panels review the potential guidelines against the criteria above and give an overall priority rating for each topic. Further information on the topic selection process can be found on the NICE Web site (4).

According to topic area, NICE refers each topic to the appropriate guideline developing center. The center then develops a scope for each guideline; this includes a description of the need for the guideline and an explanation of what will and will not be covered in terms of clinical management and the target population. Important outcome measures are also listed. Critical review of this work while in development adds rigor and provides early insight into how clinicians and patients will use the guidance. Stakeholders are consulted to determine which priority areas will most benefit from the recommendations in this topic. An independent guideline review panel, which is a standing committee recruited by NICE that is separate from the guideline development group, also assesses the scope and whether stakeholder comments have been answered adequately. Once the scope is finalized and detailed clinical questions are written, the topic is passed on to the guideline development group (see the section “Setting Review Questions”).

Stakeholders register with NICE through the Web site if they have an interest in a particular guideline topic. These stakeholders vary and may include patient and carer organizations; national organizations that represent health care professionals; clinicians; companies that manufacture the medicines or devices under discussion; providers and commissioners of health services in England, Wales and Northern Ireland; and statutory and research organizations.

All registered stakeholders (including patient and carer organizations) are involved during guideline development. They are invited to a workshop to discuss the proposed scope and comment on the draft scope that is posted on the NICE Web site during a consultation period. They also can comment on the first draft of the completed guideline and a prepublication check version of the document, both of which are posted on the NICE Web site.

The center producing the guideline recruits an independent guideline development group of health care professionals and patient and carer representatives, who are regarded as an independent advisory group to the Department of Health. Each group has 10 to 12 members, and a full mix of relevant specialties reflecting those involved in the care pathway is sought. A new group of members with expertise in the topic is convened for each guideline; members join through an open and competitive application process. Potential conflicts of interest are declared by candidates at the application stage. Technical experts in systematic reviewing, health economics, and information science support the group from the guideline center, and development is facilitated by project managers. If voting is used on a particular guideline, the technical team has a maximum of 3 votes. The technical team organizes the meetings, reviews and summarizes the evidence, and writes up the results of discussions. The clinical and patient representative members guide the technical team, examine the evidence, and make recommendations.

The target users of each guideline are outlined in the scope, and some topics will be applicable mainly to primary care or to secondary or tertiary care, for example. A version of the final recommendations is produced in language that is suitable for patients, carers, and the public.

The main development phase takes about 16 months. During this time, the literature is searched and critically appraised, the guideline development group meets every 4 to 8 weeks, and a draft guideline is produced.

The areas in the scope are developed into specific review questions. The PICO (patient, intervention, comparison, outcome) framework is usually used to structure the questions and to ensure they are specific enough that the literature search can be conducted successfully. Review questions may pertain to specific interventions or to the accuracy or clinical value (the usefulness in guiding treatment decisions and improving patient outcomes) of diagnostic tests or prognostic factors. Prognostic evidence may be helpful to the guideline development group for various purposes, such as determining risk categories or subgroups and providing information to patients.

A full systematic review is conducted for each review question. Therefore, getting the review questions correct is a crucial part of the process to ensure that the right literature is found and that the guideline development group has sufficient evidence to make recommendations.

Information scientists at the guideline center search several core databases for every guideline, depending on the study design. These databases include the Cochrane Database of Systematic Reviews, DARE, CENTRAL (Cochrane Central Register of Controlled Trials), MEDLINE, EMBASE, CINAHL, HEED, and NHS EED (National Health Service Economic Evaluation Database). Subject-specific databases and other sources of information may also be searched for certain topics. On the basis of the review questions developed by the guideline development group and the technical team, a search strategy is constructed, with the aim of being sensitive enough to retrieve all the studies of interest but precise enough not to produce an overwhelming number of results. Depending on the review area, searches may be limited by various criteria, such as date, language, study design, and participant age. Search filters may be used to aid consistency. Questions may be grouped together for the purposes of searching if they overlap. Search terms are agreed on with the guideline development group and are modified if needed. Searches are run again 6 to 8 weeks before the first draft of the guideline is finalized so that any studies published during the development period can be included. Further information can be found in chapter 5 of the NICE guidelines manual (1).

The Figure shows a diagram of the study selection process. A first sift of the titles of articles retrieved by the search is conducted to remove those that are outside the topic. Usually, several thousand titles are sifted at this stage. Abstracts are then examined to remove those that are not relevant to the review questions. From the remaining abstracts, those that do not meet the inclusion criteria determined by the guideline development group are excluded (for example, the study does not report the outcomes of interest). Because there is always potential for bias or error when selecting studies, a second reviewer performs sampling checks.

The information scientist searches for evidence that is appropriate for each review question. For review questions about the effectiveness of an intervention, a randomized, controlled trial is usually the best study design to answer the question (5). Questions on the clinical value of a diagnostic test are best answered by randomized, controlled trials and questions on the accuracy of a test are best answered by cross-sectional studies in which the index test and reference standard are performed in the same patients (6). Questions on prognosis are best answered by prospective cohort studies (7). If no studies of the desired design are found, the guideline development group considers whether studies of a weaker design will be helpful, taking into account the time available and the importance of the particular review question.

The quality of all studies that fall within the inclusion criteria is assessed by using a checklist that is appropriate to the study design. Poorly conducted studies may be excluded by agreement of the guideline development group. Studies that are considered of adequate quality are examined, and the relevant data are extracted into evidence tables, which are published in the appendix of the guideline. Where appropriate, data are pooled by using meta-analysis.

For each of the review questions on clinical effectiveness, the data are summarized by using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) system (6,8 - 11). In this system, the quality of evidence is assessed for each relevant outcome and is based on study design, limitations, and consistency and directness of the evidence, which when making recommendations enables the guideline development group to interpret the quality of the evidence and therefore the confidence by which the group then can make a clinical recommendation.

Studies of cost-effectiveness are also assessed against a checklist. A systematic review is conducted for economic studies in all areas covered by the guideline. If the health economic literature is incomplete or inconclusive, the guideline development group selects priority areas for further economic modeling by considering various factors, including the number of patients affected, the potential effect on patient outcomes and costs, and the extent to which a new economic analysis may reduce uncertainty about cost-effectiveness.

All economic models are constructed by the health economist in collaboration with the guideline development group and are based on the best evidence from the guideline's systematic review of the clinical evidence. The results are summarized in the full version of the guideline, along with the data used to input into models constructed by the health economist. Further details on the methods used to assess cost-effectiveness will be described in a future article and in the NICE guidelines manual (1); detailed methods sections for the published full versions of the guidelines will be available at www.ncgc.ac.uk.

The guideline development group uses the GRADE approach to consider clinical and cost-effectiveness evidence before deciding on the best recommendation to make. The quality of the evidence underpinning the recommendations is examined, and the strength of the recommendation is debated by the guideline development group, which examines its clinical significance and, through interpretation of the evidence, chooses wording that expresses the strength of the recommendation.

Recommendations stating that an intervention “must” be used are usually made only when there is a legal obligation to do so or when the consequences of not complying with the recommendation are very serious. For example, “Ultra-rapid detoxification under general anaesthesia or heavy sedation (where the airway needs to be supported) must not be used. This is because of the risk of serious adverse events, including death.” If the guideline development group is confident that a recommendation will do more good than harm for most patients and that the intervention is likely to be cost-effective, a “strong” recommendation is made that an intervention be used: “Offer an alpha blocker (alfuzosin, doxazosin, tamsulosin or terazosin) to men with moderate to severe lower urinary tract symptoms.” Likewise, if the guideline development group believes that an intervention does more harm than good or is less cost-effective than an alternative treatment, it may choose to make a recommendation that an intervention should not be used: for example, “Do not offer a tilt test to people who have a diagnosis of vasovagal syncope.” Both clinicians and patients have indicated that this direct wording is preferred when translating recommendations into clinical practice in order to produce sustainable change.

If an intervention could be used because the guideline development group is certain that the intervention will do more good than harm for most patients, but other options that are similarly cost-effective are available or some patients may opt for a less effective but cheaper treatment, a recommendation may be made to consider the intervention; for example, “Consider a serum natriuretic peptide test (if not already performed) when heart failure is still suspected after transthoracic Doppler 2D echocardiography has shown a preserved left ventricular ejection fraction.” This type of recommendation may be made if the balance between risks and benefits is close or if certain groups of patients may be more suitable for a different treatment (for example, among persons who are particularly averse to certain side effects).

Recommendations have not been given a grade since 2006 because of concerns that the grading systems available at that time confused the quality of evidence with the strength of recommendation, and because of anecdotal evidence that recommendations that were important but had lower grades were being ignored.

Formal or informal consensus methods are used to help make recommendations in all circumstances. Formal methods are more likely to be used where there is a lack of evidence or disagreement among the guideline development group, or to finalize the recommendations. Voting is occasionally used to help the group make its final decisions. Recommendations generally go through several iterations before they are finalized. This process is usually conducted through face-to-face meetings and online discussions. If the guideline development group considers the evidence to be lacking in an area, it may choose to make a research recommendation.

The deliberations of the guideline development group are summarized in the full version of the guideline, in a section describing the link between evidence and recommendations. Every recommendation includes discussion of the relative value of the different outcomes that were used to make the recommendation, the trade-off that the guideline development group made between the benefits and harms of the intervention, economic considerations, the quality of the evidence, and other considerations (including equality issues, implementation considerations, patient preferences, or anything that the guideline development group believes is important).

All NICE guidelines have a validation phase that involves a consultation period for stakeholders. This important process allows for peer review and quality assurance. For new guidelines, external review usually takes 8 weeks (or 4 weeks for a short guideline); updated guidelines may have a shorter consultation period, depending on the scope of the update.

Stakeholders are invited to submit comments on the first draft of the guideline, which is posted on the NICE Web site. The guideline center and the guideline development group respond to all comments and make changes to the guideline as needed. The National Institute for Health and Clinical Excellence also commissions a review of the health economic and statistics in every guideline. A guideline review panel convened by NICE reviews the responses to the stakeholder comments to check that they have all been answered appropriately and that the scope of the guideline has been covered.

Technical experts at NICE and the guideline review panel examine guidelines from all of the guideline-producing centers and are therefore able to ensure consistency of development process and quality of output. Before publication, stakeholders are invited to check for any factual errors or inaccuracies in the full version of the guideline. Any corrections are made, and the final guideline is published on the NICE Web site.

A check for new evidence is made 3 years after publication, unless the need for an earlier update has been identified. Health care professionals and patients are also approached to see whether there have been any changes in practice or new evidence has emerged that might mean the guideline should be updated. With this information, NICE decides whether to commission a full update, a partial update, or no update.

Each guideline is produced in 4 formats, each of which is suitable for different audiences. The full version details all methods, evidence, and the link between evidence and recommendations; the NICE version lists the recommendations; the quick reference guide summarizes the recommendations for health care professionals; and the “understanding NICE guidance” version summarizes the guideline for patients, carers, and the general public.

The wording of each recommendation is considered carefully to ensure that it is clear and concise. Recommendations are action-based and attempt to be specific about the intervention and the population to which the recommendation applies, so that it is clear how they translate into clinical practice. Justifications for recommendations are not normally included because this information can be found in a section describing the link between evidence and recommendations in the full version of the guideline described above.

The guideline development group selects 5 to 10 recommendations as key priorities for implementation. These are not necessarily all “strong” recommendations; rather, they are selected as being likely to do at least 1 of the following: have a high impact on outcomes that are important to patients; have a high impact on reducing variation in care and outcomes; lead to more efficient use of National Health Service resources; promote patient choice; or promote more equitable access to health care.

In addition, the guideline development group identifies recommendations that are likely to benefit from support in implementation. Criteria include whether a recommendation relates to an intervention that is not part of routine care, requires changes in service delivery, requires retraining of staff or the development of new skills and competencies, highlights the need for practice to change, would have to be implemented across several agencies or settings (complex interactions), may be viewed as potentially contentious, or may be difficult to implement for other reasons. These key recommendations are clearly identified near the beginning of the guideline documents.

While assessing the evidence and making recommendations for best practice, the guideline development group also considers how the recommendations will be implemented. The group may comment on important aspects in the section of the full version of the guideline and further explain the link between evidence and recommendations.

The National Institute for Health and Clinical Excellence has set up a team to assist with implementation of the guidelines. This team helps disseminate the guidelines to the appropriate audiences and provides tools to help put the guidelines into practice. The tools developed for each guideline include costing tools, such as the national cost reports and costing spreadsheet that allow assessment of how the guidance will affect local budgets; audit criteria and support; a slide set containing the key messages from the guideline that can be tailored for local use; and other support tools, including practical suggestions on local implementation of specific pieces of guidance, reviews of the policy context, key drivers, probable barriers to implementation and possible ways to overcome these, and educational tools.

In the past year, NICE has developed quality standards, which are statements that act as markers of high-quality, cost-effective patient care. Quality standards are based on the best available evidence, using clinical guidelines if they exist. The standards are not limited to using only “strong” recommendations as their basis; other NHS Evidence–accredited sources may also be used if required. The methods for producing quality standards are currently being developed in light of feedback from the first few pilot topics.

The quality standards can be used in many ways, including in national audit, to monitor quality improvements, to show that high-quality care is being provided, to highlight areas for improvement, to provide commissioners with assurance that high-quality care is being provided, or to provide incentives for a provider. They can be used by patients and the public to find out about the quality of care they can expect to receive from their health care provider.

The National Institute for Health and Clinical Excellence provides funding for the National Clinical Guideline Centre and the other guideline-developing centers. However, the guideline development group develops the draft guidelines independently of NICE. Representatives from NICE may attend guideline development group meetings as observers. During the validation process for the guideline, NICE conducts assurance checks that correct processes have been followed and that the guideline meets expected standards of quality. The guidance executive for NICE signs off the guideline recommendations before they go forward for publication.

All members of the guideline development group and the staff of the guideline center declare any potential conflicts of interest for the guidelines they are involved in. Interests are declared at every meeting and are categorized as personal pecuniary, family pecuniary, nonpersonal pecuniary, and personal nonpecuniary interests. Guideline development group members are not permitted to take part in discussions relating to areas for which they have a serious conflict of interest; such discussions include those regarding evidence, drafting of specific recommendations, and writing particular sections of the guideline documents. Decisions about whether a conflict of interest exists are made by the chair of the guideline and the director of the guideline center. The staff of the guideline cannot work on a guideline area for which they have a conflict of interest. These declarations of interest are published in the minutes of the meeting and in the full version of the final guideline. Further information on the conflicts of interest policy and all other aspects of the development process can be found on the NICE Web site (www.nice.org.uk).