0
Clinical Guidelines |

Guidelines International Network: Toward International Standards for Clinical Practice Guidelines FREE

Amir Qaseem, MD, PhD, MHA; Frode Forland, MD, DPH; Fergus Macbeth, MD; Günter Ollenschläger, MD, PharmD, PhD; Sue Phillips, PhD; Philip van der Wees, PhD, PT, for the Board of Trustees of the Guidelines International Network
[+] Article and Author Information

Disclaimer: The 2010–2011 G-I-N board of trustees is responsible for the content of this article. This article does not necessarily reflect the views or policies of the membership of G-I-N.

Acknowledgment: The authors thank the 2010–2011 G-I-N board of trustees.

Financial Support: From G-I-N's operating budget.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-2693.

Requests for Single Reprints: Amir Qaseem, MD, PhD, MHA, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106; e-mail, mailto:aqaseem@acponline.org.

Current Author Addresses: Dr. Qaseem: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.

Dr. Forland: KIT Biomedical Research, Meibergdreef 39, 1105 AZ Amsterdam, the Netherlands.

Dr. Macbeth: National Institute for Health and Clinical Excellence, MidCity Place, 71 High Holborn, London WC1V 6NA, United Kingdom.

Dr. Ollenschläger: Ärztliches Zentrum für Qualität in der Medizin, Tiergarten Tower, Straße des 17, Juni 106-108, 10623 Berlin, Germany.

Dr. Phillips: National Health and Medical Research Council, GPO Box 4530, Melbourne, Victoria 3004, Australia.

Dr. van der Wees: Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA 02115-5899.

Author Contributions: Conception and design: A. Qaseem, F. Forland, F. Macbeth, P. van der Wees.

Analysis and interpretation of the data: A. Qaseem, F. Forland, P. van der Wees.

Drafting of the article: A. Qaseem, F. Forland, F. Macbeth, S. Phillips, P. van der Wees.

Critical revision of the article for important intellectual content: A. Qaseem, F. Forland, F. Macbeth, G. Ollenschläger, S. Phillips, P. van der Wees.

Final approval of the article: A. Qaseem, F. Forland, F. Macbeth, G. Ollenschläger, S. Phillips, P. van der Wees.

Administrative, technical, or logistic support: A. Qaseem.

Collection and assembly of data: A. Qaseem.

* For a list of members of the board of trustees of the Guidelines International Network, see the Appendix.


From the American College of Physicians, Philadelphia, Pennsylvania; KIT Biomedical Research, Royal Tropical Institute, Amsterdam, the Netherlands; National Institute for Health and Clinical Excellence, London, United Kingdom; Ärztliches Zentrum für Qualität in der Medizin, Berlin, Germany; National Health and Medical Research Council, Melbourne, Australia; Maastricht University, Maastricht, Radboud University Nijmegen Medical Centre, Nimejen, Royal Dutch Society for Physical Therapy, Amersfoot, the Netherlands; and Harvard Medical School, Boston, Massachusetts.


Ann Intern Med. 2012;156(7):525-531. doi:10.7326/0003-4819-156-7-201204030-00009
Text Size: A A A

Reader Survey: Are standards needed for clinical guideline development? http://www.annals.org/content/156/7/525/suppl/DC1 Tell us what you think.Guideline development processes vary substantially, and many guidelines do not meet basic quality criteria. Standards for guideline development can help organizations ensure that recommendations are evidence-based and can help users identify high-quality guidelines. Such organizations as the U.S. Institute of Medicine and the United Kingdom's National Institute for Health and Clinical Excellence have developed recommendations to define trustworthy guidelines within their locales. Many groups charged with guideline development find the lengthy list of standards developed by such organizations to be aspirational but infeasible to follow in entirety.Founded in 2002, the Guidelines International Network (G-I-N) is a network of guideline developers that includes 93 organizations and 89 individual members representing 46 countries. The G-I-N board of trustees recognized the importance of guideline development processes that are both rigorous and feasible even for modestly funded groups to implement and initiated an effort toward consensus about minimum standards for high-quality guidelines. In contrast to other existing standards for guideline development at national or local levels, the key components proposed by G-I-N will represent the consensus of an international, multidisciplinary group of active guideline developers.This article presents G-I-N's proposed set of key components for guideline development. These key components address panel composition, decision-making process, conflicts of interest, guideline objective, development methods, evidence review, basis of recommendations, ratings of evidence and recommendations, guideline review, updating processes, and funding. It is hoped that this article promotes discussion and eventual agreement on a set of international standards for guideline development.


The health care profession relies heavily on the translation of evidence into clinical practice guidelines (1). The U.S. Institute of Medicine (IOM) defines clinical practice guidelines as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” (2). Over recent decades, the number of guidelines developed by government and private organizations worldwide has increased exponentially. Clinicians, patients, and other stakeholders struggle with numerous and sometimes contradictory guidelines of variable quality (3).

Development of guidelines within coordinated programs can facilitate meeting quality standards by enabling the efficient sharing of resources and expertise (4). International collaboration offers additional opportunities to enhance guideline development (4). Standards for guideline development can help organizations assure that recommendations are evidence-based and can help users identify high-quality guidelines. Although the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument does not explicitly set standards for guideline development, it is a valuable tool to evaluate the process of practice guideline development (4).

Several groups, such as the IOM (2), World Health Organization (5), National Institute for Health and Clinical Excellence (6), Scottish Intercollegiate Guidelines Network (7), National Health and Medical Research Council (8), many medical societies (915), and others (1624), have proposed standards for guideline developers. Of note, the IOM's recent reports identifying criteria for trustworthy clinical practice guidelines and systematic reviews (2, 25) have received both praise and criticism. Much of the concern about the IOM's criteria centers on the feasibility of implementing the long list of criteria and the applicability to diverse settings (26).

Founded in 2002, the Guidelines International Network (G-I-N) (www.g-i-n.net) is a network of guideline developers composed of 93 organizations and 89 individual members representing 46 countries (as of January 2012) (27). Its online library currently comprises more than 7400 documents, including 3636 guidelines, with a wide range of variation in quality. The Guidelines International Network understands the critical need to minimize the quality differences among guidelines and to promote the development of trustworthy guidelines. In response to calls for international standards to help develop and appraise clinical guidelines (19, 2830), the G-I-N board of trustees reviewed the current literature and used a consensus process to propose a set of key components for guideline development. The intent is to initiate global discussion and consensus about minimal standards for guideline development.

The G-I-N board of trustees includes clinicians and guideline developers with specific skills in evidence-based medicine and guideline development and implementation. The board represents diverse geographic locales, including North America, Europe, Australia, and Asia. The authors of this article are members of the G-I-N board of trustees who were not selected to represent particular organizations.

A search of PubMed, Web sites of organizations that develop guidelines, Google, and Google Scholar identified relevant manuals, protocols, and published articles about guideline development (2, 47, 925, 27115). We searched reference lists of identified publications and documents for further sources of information. We applied no language restriction and searched through October 2011.

We identified components of guideline development critical to the development of high-quality guidelines. We also considered practical issues related to incorporating these key components into the guideline development process, as well as issues related to implementation and global adaptation. Only components for guideline development that were deemed essential were used to generate the list of key elements described in this article. We selected the key components by informal consensus, and the article was presented to the full G-I-N board of trustees for approval. Because the purpose of this article is to present components to initiate a debate for the future revision or development of standards for guideline development, we considered using an informal approach to allow inclusion of different perspectives and opinions.

Key Components of a High-Quality and Trustworthy Guideline

Development of clinical guidelines involves several steps that can each be executed with differing degrees of rigor. We believe that the following 11 key components are important minimal criteria for high-quality guidelines. The Table presents an overview of the criteria. The Guidelines International Network recommends that guideline developers strive to meet these criteria and recognize that adaptation to local circumstances may be necessary and appropriate. Guideline development organizations should specify how they put each of these key components into effect in documents that detail their methods for guideline development.

Table Jump Placeholder

Table. Key Components of High-Quality and Trustworthy Guidelines

1. Composition of Guideline Development Group

A guideline development panel should include diverse and relevant stakeholders, such as health professionals, methodologists, experts on a topic, and patients or other health care consumers.

The guideline development group is responsible for reviewing the evidence, translating it into practice recommendations, writing the guideline, and assuring that the recommendations are not biased by being based on factors other than the best available scientific evidence. Groups without multidisciplinary membership can have been associated with recommendations that are not evidence-based (53, 72, 75, 80, 102). Thus, guideline development groups should include diverse stakeholders, such as health professionals; content experts; methodologists with skills in evidence appraisal and synthesis; and, ideally, health care consumers and health economists.

A dysfunctional group may also yield unreliable recommendations (90). Therefore, an effective and neutral chair should lead the group to ensure balanced contributions from all members. The primary role of the chair is to facilitate discussion and consensus. The chair should have general knowledge of the topic but does not need be a topic expert. In fact, a chair with topic expertise creates the risk that the chair's preconceived opinions could bias deliberations.

The size of the guideline development group is also important. Large groups may be difficult to manage, and small groups may lack relevant stakeholders. Although no good evidence supports the appropriate size of the guideline development group (2), our experience suggests that guideline panels of 10 to 20 persons usually work well.

Whether and how to best involve health care consumers in the guideline development process is a topic of debate (77, 84, 98, 116). Consumers' views about the quality of life experienced with different medical conditions and interventions can be valuable (77). However, lack of training in evidence-based medicine and limited scientific literacy can hinder an evidence-based process. If consumers are included as voting members of guideline panels, consumers may need training and support to fulfill their role. To help understand how to effectively include patients in the guideline development process, G-I-N PUBLIC (www.g-i-n.net/activities/gin-public) was formed to develop strategies to aid guideline developers to effectively engage patients, consumers, and their families.

2. Decision-Making Process

A guideline should describe the process used to reach consensus among the panel members and, if applicable, approval by the sponsoring organization. This process should be established before the start of guideline development.

Guideline development, even when evidence-based, is a process that involves group consensus. Consensus is required to select and interpret evidence, translate evidence into recommendations, and determine how to handle situations when evidence is lacking to answer important clinical questions. Although some countries and organizations use formal consensus processes, many organizations use informal processes.

Examples of formal processes include the nominal group technique, Delphi, or formal balloting (50, 52, 109). “Informal process” means using no structured methods to come to a consensus. Formal methods have been shown to result in a less biased and more evidence-based process than informal methods (72, 80, 100). Regardless of the process, the guideline should clearly define a quorum and document the consensus process (50, 52, 109).

3. Conflicts of Interest

A guideline should include disclosure of the financial and nonfinancial conflicts of interest for members of the guideline development group. The guideline should also describe how any identified conflicts were recorded and resolved.

Conflict of interest (COI) disclosure and management policies of various guideline developing organizations vary widely (2). Conflicts of interest are “a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by secondary interest” (117). Members of a guideline development group should disclose any personal or household financial and nonfinancial COI relationships related to the guideline topic. If a member or someone in the member's household has a relevant potential financial, nonfinancial, professional, or other personal gain or loss associated with the topic of a guideline, any such conflicts should be disclosed and clearly stated in a guideline document.

Financial COI includes ownership of stocks or shares, paid employment or consultancy, paid board memberships, patent applications, research grants (from any source, whether restricted or unrestricted), honoraria, and gifts. Nonfinancial COI includes leadership or board or committee memberships, involvement with an advocacy group that may gain from a guideline, writing or consulting for an educational company, or having personal convictions (political, religious, ideological, or other) related to the guideline topic that may interfere with an unbiased evidence review or recommendation process. The guideline development group and sponsoring organization must actively and transparently manage COIs by assessing the level of risk and, if necessary, excluding the member with the COI from relevant discussions and decisions.

4. Scope of a Guideline

A guideline should specify its objective(s) and scope.

Guidelines should clearly state their objectives and the key questions that they address. The scope includes diagnostic criteria, benefits and harms of various treatment options, key outcomes that have been evaluated, target patient population, and intended users of the guideline (62, 101).

5. Methods

A guideline should clearly describe the methods used for the guideline development in detail.

A clear description of the development process should accompany all guidelines, either within the guideline document or in a separate, referenced document (118). The description of guideline development methods should reflect the key components as presented in this article and includes the process for choosing group members and a chair, methods of reviewing evidence, the process that the group used to deliberate about the evidence and formulate recommendations, dissemination and implementation of the guideline, and any pertinent review or approval processes.

6. Evidence Reviews

Guideline developers should use systematic evidence review methods to identify and evaluate evidence related to the guideline topic.

Most experts agree that trustworthy guidelines are based on high-quality systematic reviews of evidence (2, 51, 103, 119). Systematic reviews use rigorous methods to identify clinical questions, inclusion and exclusion criteria, and methods for rating the quality of available evidence. The Guidelines International Network has developed templates for summarizing studies addressing diagnostic and intervention questions, and templates for prognostic studies and health economics assessments are under development (22). There are papers, including the recent IOM report, that describe the standards for a good systematic review in detail (25, 120).

7. Guideline Recommendations

A guideline recommendation should be clearly stated and based on scientific evidence of benefits; harms; and, if possible, costs.

Guideline recommendations should be clear, evidence-based statements that aim to provide guideline users with clear directions for effective delivery of care. The recommendations should be supported by careful consideration of evidence; quantification of the magnitude of benefits and harms, as well as costs when possible; resource and feasibility issues; implementation considerations; patient and caregiver preferences and concerns; and ethical and legal matters. Recommendations related to interventions should use unambiguous, active language that reflects the strength of the evidence. A recommendation should be actionable and use the active voice (121). Guideline developers should aim to use such terms as “should” or “recommend” and to avoid using such vague words and phrases as “may,” “can,” or “consider,” unless real uncertainty exists about the evidence effectiveness, because these terms are not helpful for practical implementation (122).

8. Rating of Evidence and Recommendations

A guideline should use a rating system to communicate the quality and reliability of both the evidence and the strength of its recommendations.

Guideline developers should synthesize and grade evidence by using a standardized approach. The strength of recommendations should be assigned on the basis of evaluation of the evidence, benefits and harms, consistency, clinical effect, and generalizability and applicability, as well as patient preferences. Clear identification of the quality of evidence helps, and strength of clinical recommendations increases the trustworthiness and improves the implementation of clinical guidelines (32, 43, 76). Several grading systems are currently available, including the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system, which is increasingly being adopted by guideline developers worldwide (34, 66).

9. Peer Review and Stakeholder Consultations

Review by external stakeholders should be conducted before guideline publication.

Guidelines should be reviewed by stakeholders external to the guideline group before publication. Reviewers may include outside experts, representatives of the sponsoring organization, and members of the public (4, 101, 105). The review should include not only content-related review but also methodological review of both the evidence report and the guideline. When selecting peer reviewers, it is important to consider those who are more likely to provide comments based on scientific and clinical knowledge rather than unsubstantiated views (101). A summary of the external review process should accompany a guideline.

10. Guideline Expiration and Updating

A guideline should include an expiration date and/or describe the process that the guideline groups will use to update recommendations.

Guidelines become outdated at different rates depending on the availability of new evidence. Therefore, it is important to identify the expiration date of a guideline, as well as an update process, if planned. Developers should prospectively determine whether and when they will update a guideline or when it should be considered inactive if an update is not performed.

11. Financial Support and Sponsoring Organization

A guideline should disclose financial support for the development of both the evidence review as well as the guideline recommendations.

Guidelines should identify the sponsoring organization and its role in the development of a clinical guideline. In addition, any honoraria or financial support provided to the authors of a guideline should also be fully disclosed.

The Guidelines International Network's goal in proposing these minimum standards is to promote the development of high-quality guidelines that serve patients well. As a central repository for all clinical guidelines of our member organizations, it is critical for G-I-N to ensure that our members consider our guidelines library as a trustworthy and valuable resource. We currently do not require guidelines to meet our proposed minimum criteria in order to be listed in our library. However, the proposed key components presented in this article should help guideline developers and users assess the strengths and weaknesses of a guideline and thus clearly indicate which guidelines can be considered trustworthy.

We hope that this article will promote discussion and possible agreement among a broad array of guideline developers, although we recognize that small variations at the local level may be inevitable and appropriate. The Guidelines International Network provides a platform for international discussion and will use the organization's infrastructure to promote discussion among our members toward globally endorsed minimum standards for guideline development.

Qaseem A, Snow V, Gosfield A, Gregg D, Michl K, Wennberg D. et al.  Pay for performance through the lens of medical professionalism. Ann Intern Med. 2010; 152.366-9
PubMed
 
Institute of Medicine.  Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Pr; 2011.
 
Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM.  Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003; 139.493-8
PubMed
 
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE Next Steps Consortium.  Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010; 182.472-8
PubMed
CrossRef
 
World Health Organization.  Global Programme on Evidence for Health Policy. Geneva: World Health Organization; 2003: Accessed at http://whqlibdoc.who.int/hq/2003/EIP_GPE_EQC_2003_1.pdf on 1 October 2011.
 
The National Institute for Health and Clinical Excellence.  The Guidelines Manual (January 2009). London; 2009. Accessed at www.nice.org.uk/guidelinesmanual on 1 October 2011.
 
Scottish Intercollegiate Guidelines Network.  SIGN 50: A Guideline Developer's Handbook. Edinburgh, Scotland: Scottish Intercollegiate Guidelines Network; 2008. Accessed at www.sign.ac.uk/guidelines/fulltext/50/index.html on 1 October 2011.
 
National Health and Medical Research Council.  Procedures and Requirements for Meeting the 2011 NHMRC Standard for Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2011.
 
American Society of Clinical Oncology.  American Society of Clinical Oncology Guideline Procedures Manual. Expert Panel Version 4.0. Alexandria, VA: American Soc Clinical Oncology; 2011. Accessed at www.asco.org/ASCO/Downloads/Cancer%20Policy%20and%20Clinical%20Affairs/Clinical%20Affairs%20(derivative%20products)/Manual/Methodology%20Manual%201.25.11.pdf on 1 October 2011.
 
American College of Cardiology Foundation and American Heart Association.  Methodology Manual and Policies from the ACCF/AHA Task Force on Practice Guidelines. 2010. Accessed at http://my.americanheart.org/professional/StatementsGuidelines/PoliciesDevelopment/Development/Methodologies-and-Policies-from-the-ACCAHA-Task-Force-on-Practice-Guidelines_UCM_320470_Article.jsp on 1 October 2011.
 
Baumann MH, Lewis SZ, Gutterman D, American College of Chest Physicians.  ACCP evidence-based guideline development: a successful and transparent approach addressing conflict of interest, funding, and patient-centered recommendations. Chest. 2007; 132.1015-24
PubMed
 
Edlund W, Gronseth G, Yuen S, Franklin G; Quality Standards Subcommittee and Therapeutics and Technology Assessment Subcommittee.  American Academy of Neurology Clinical Practice Guideline Process Manual. 2004. Accessed at www.aan.com/globals/axon/assets/3749.pdf on 1 October 2011.
 
 Attributes of ATS documents that guide clinical practice. Recommendations of the ATS Clinical Practice Committee. Am J Respir Crit Care Med. 1997; 156.2015-25
PubMed
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153.194-9
PubMed
 
The Intensive Care Society.  SSQ Committee Guideline Policy. 2010. Accessed at www.ics.ac.uk/intensive_care_professional/standards_and_guidelines/ssq_guideline_policy on 1 October 2011.
 
Committee of Ministers of the Council of Europe.  Developing a Metholology for Drawing up Guidelines on Best Medical Practices: Recommendation Rec(2001)13. Strasbourg, Cedex, France: Council of Europe Publishing; 2001. Accessed at www.leitlinien.de/mdb/edocs/pdf/literatur/coe-rec-2001-13.pdf on 1 October 2011.
 
Dutch Institute for Healthcare Improvement CBO.  Accessed at www.cbo.nl/en/ on 1 October 2011.
 
German Agency for Quality in Medicine.  Accessed at www.aezq.de/ on 1 October 2011.
 
Grol R, Cluzeau FA, Burgers JS.  Clinical practice guidelines: towards better quality guidelines and increased international collaboration [Editorial]. Br J Cancer. 2003; 89.Suppl 1S4-8
PubMed
 
Guidelines International Network.  Development and Training Resources. Accessed at www.g-i-n.net/library/development-and-training-resources on 1 October 2011.
 
McDonald CJ, Overhage JM.  Guidelines you can follow and can trust. An ideal and an example [Editorial]. JAMA. 1994; 271.872-3
PubMed
 
Mlika-Cabanne N, Harbour R, de Beer H, Laurence M, Cook R, Twaddle S, Guidelines International Network (GIN) Working Group on Evidence Tables.  Sharing hard labour: developing a standard template for data summaries in guideline development. BMJ Qual Saf. 2011; 20.141-5
PubMed
 
Rosenfeld RM, Shiffman RN.  Clinical practice guidelines: a manual for developing evidence-based guidelines to facilitate performance measurement and quality improvement. Otolaryngol Head Neck Surg. 2006; 135.S1-28
PubMed
 
Rosenfeld RM, Shiffman RN.  Clinical practice guideline development manual: a quality-driven approach for translating evidence into action. Otolaryngol Head Neck Surg. 2009; 140.S1-43
PubMed
 
Institute of Medicine.  Finding What Works in Healthcare. Washington, DC: National Academies Pr; 2011.
 
Kuehn BM.  IOM sets out “gold standard” practices for creating guidelines, systematic reviews. JAMA. 2011; 305.1846-8
PubMed
 
Ollenschläger G, Marshall C, Qureshi S, Rosenbrand K, Burgers J, Mäkelä M, et al. Board of Trustees 2002, Guidelines International Network (G-I-N).  Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004; 13.455-60
PubMed
 
Grilli R, Magrini N, Penna A, Mura G, Liberati A.  Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000; 355.103-6
PubMed
 
Shaneyfelt TM, Centor RM.  Reassessment of clinical practice guidelines: go gently into that good night [Editorial]. JAMA. 2009; 301.868-9
PubMed
 
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J.  Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999; 281.1900-5
PubMed
 
AGREE Collaboration.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003; 12.18-23
PubMed
 
Ansari MT, Tsertsvadze A, Moher D.  Grading quality of evidence and strength of recommendations: a perspective. PLoS Med. 2009; 6.1000151
PubMed
 
Appraisal of Guidelines for Research and Evaluation Instrument.  2001. Accessed at www.agreetrust.org/ on 1 October 2011.
 
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004; 328.1490
PubMed
 
Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al. GRADE Working Group.  Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005; 5.25
PubMed
 
Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. GRADE Working Group.  Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004; 4.38
PubMed
 
Browman GP.  Improving clinical practice guidelines for the 21st century. Attitudinal barriers and not technology are the main challenges. Int J Technol Assess Health Care. 2000; 16.959-68
PubMed
 
Browman GP.  Evidence-based clinical practice guideline development: principles, challenges, and accountability to evidence [Editorial]. J Surg Oncol. 2010; 101.1-2
PubMed
 
Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J, AGREE Collaboration.  Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003; 15.31-45
PubMed
 
Burgers JS, Grol RP, Zaat JO, Spies TH, van der Bij AK, Mokkink HG.  Characteristics of effective clinical guidelines for general practice. Br J Gen Pract. 2003; 53.15-9
PubMed
 
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA. et al.  Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999; 282.1458-65
PubMed
 
Calderón C, Rotaeche R, Etxebarria A, Marzo M, Rico R, Barandiaran M.  Gaining insight into the Clinical Practice Guideline development processes: qualitative study in a workshop to implement the GRADE proposal in Spain. BMC Health Serv Res. 2006; 6.138
PubMed
 
Calogne N, Harris R.  United States Preventive Services Task Force (USPSTF). Presented at Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Irvine, California, 19 April 2010.
 
Carter AO, Battista RN, Hodge MJ, Lewis S, Basinski A, Davis D.  Report on activities and attitudes of organizations active in the clinical practice guidelines field. CMAJ. 1995; 153.901-7
PubMed
 
Christiaens T, De Backer D, Burgers J, Baerheim A.  Guidelines, evidence, and cultural factors. Scand J Prim Health Care. 2004; 22.141-5
PubMed
 
Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR.  The relation between systematic reviews and practice guidelines. Ann Intern Med. 1997; 127.210-6
PubMed
 
Davis D, Goldman J, Palda V.  Handbook on Clinical Practice Guidelines. Toronto, Canada: Canadian Medical Assoc; 2007.
 
Eccles M, Freemantle N, Mason J, Using systematic reviews in clinical guideline development.Ross AC, Taylor CL, Yaktine AL, Del Valle HB.  Systematic Reviews in Health Care: Meta-Analysis in Context. 2nd ed. Systematic Reviews in Health Care. London: BMJ Publishing Group; 2008.400
 
Fervers B, Burgers JS, Haugh MC, Brouwers M, Browman G, Cluzeau F. et al.  Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care. 2005; 17.123-32
PubMed
 
Fink A, Kosecoff J, Chassin M, Brook RH.  Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984; 74.979-83
PubMed
 
Freeman AC, Sweeney K.  Why general practitioners do not implement evidence: qualitative study. BMJ. 2001; 323.1100-2
PubMed
 
Fretheim A, Schünemann HJ, Oxman AD.  Improving the use of research evidence in guideline development: 5. Group processes. Health Res Policy Syst. 2006; 4.17
PubMed
 
Fretheim A, Schünemann HJ, Oxman AD.  Improving the use of research evidence in guideline development: 3. Group composition and consultation process. Health Res Policy Syst. 2006; 4.15
PubMed
 
Gabbay J, le May A.  Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care. BMJ. 2004; 329.1013
PubMed
 
Garfield FB, Garfield JM.  Clinical judgment and clinical practice guidelines. Int J Technol Assess Health Care. 2000; 16.1050-60
PubMed
 
Giacomini MK.  The rocky road: qualitative research as evidence. Evid Based Med. 2001; 6.4-6
 
Graham ID, Harrison MB.  Evaluation and adaptation of clinical practice guidelines. Evid Based Nurs. 2005; 8.68-72
PubMed
 
Green J, Britten N.  Qualitative research and evidence based medicine. BMJ. 1998; 316.1230-2
PubMed
 
Greenhalgh T.  “Is my practice evidence-based?” [Editorial]. BMJ. 1996; 313.957-8
PubMed
 
Griffiths F, Green E, Tsouroufli M.  The nature of medical evidence and its inherent uncertainty for the clinical consultation: qualitative study. BMJ. 2005; 330.511
PubMed
 
Grimshaw J, Eccles M, Russell I.  Developing clinically valid practice guidelines. J Eval Clin Pract. 1995; 1.37-48
PubMed
 
Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I. et al.  Developing and implementing clinical practice guidelines. Qual Health Care. 1995; 4.55-64
PubMed
 
Grimshaw JM, Russell IT.  Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993; 342.1317-22
PubMed
 
Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H.  Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998; 317.858-61
PubMed
 
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004; 328.1490
PubMed
 
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE Working Group.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336.924-6
PubMed
 
Haines A, Jones R.  Implementing findings of research. BMJ. 1994; 308.1488-92
PubMed
 
Harris JS, Sinnott PL, Holland JP, Ording J, Turkelson C, Weiss M. et al.  Methodology to update the practice recommendations in the American College of Occupational and Environmental Medicine's Occupational Medicine Practice Guidelines, second edition. J Occup Environ Med. 2008; 50.282-95
PubMed
 
Harrison MB, Légaré F, Graham ID, Fervers B.  Adapting clinical practice guidelines to local context and assessing barriers to their use. CMAJ. 2010; 182.78-84
PubMed
 
Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB.  More informative abstracts of articles describing clinical practice guidelines. Ann Intern Med. 1993; 118.731-7
PubMed
 
Heathfield H, Pitty D, Hanka R.  Evaluating information technology in health care: barriers and challenges. BMJ. 1998; 316.1959-61
PubMed
 
Hutchings A, Raine R.  A systematic review of factors affecting the judgments produced by formal consensus development methods in health care. J Health Serv Res Policy. 2006; 11.172-9
PubMed
 
Institute of Medicine.  Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Pr; 2009.
 
Jaime J.  Introduction: Practice Guidelines. Helpful Aids or Paradigm Shift? Guideline Development Methods: Information for National Collaborating Centres and Guideline Developers. Int J Technol Assess Health Care. 2005; 16.957-8
 
Kahan JP, Park RE, Leape LL, Bernstein SJ, Hilborne LH, Parker L. et al.  Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures. Med Care. 1996; 34.512-23
PubMed
 
Kavanagh BP.  The GRADE system for rating clinical guidelines. PLoS Med. 2009; 6.1000094
PubMed
 
Krahn M, Naglie G.  The next step in guideline development: incorporating patient preferences. JAMA. 2008; 300.436-8
PubMed
 
Lewis S.  Paradox, process and perception: the role of organizations in clinical practice guidelines development. CMAJ. 1995; 153.1073-7
PubMed
 
Moreira T.  Diversity in clinical guidelines: the role of repertoires of evaluation. Soc Sci Med. 2005; 60.1975-85
PubMed
 
Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P.  Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess. 1998; 2.iii-ix, 1-274
PubMed
 
National Health and Medical Research Council.  A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 1999.
 
National Health and Medical Research Council.  NHMRC Levels of Evidence and Grades for Recommendations for Developers of Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2009.
 
www.nhmrc.gov.au/guidelines/procedures-and-requirements-meeting-2011-nhmrc-standard-clinical-practice-guidelinesNational Health and Medical Research Council.  Procedures and Requirements for Meeting the 2011 NHMRC Standard for Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2011.
 
Owens DK.  Improving practice guidelines with patient-specific recommendations [Editorial]. Ann Intern Med. 2011; 154.638-9
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst. 2006; 4.20
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 7. Deciding what evidence to include. Health Res Policy Syst. 2006; 4.19
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst.. 2006; 4.14
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A, Subcommittee on the Use of Research Evidence.  Improving the use of research evidence in guideline development. Report no. 11-2007. Norwegian Knowledge Centre for the Health Services; 2007.
 
Pagliari C, Grimshaw J.  Impact of group structure and process on multidisciplinary evidence-based guideline development: an observational study. J Eval Clin Pract. 2002; 8.145-53
PubMed
 
Pagliari C, Grimshaw J, Eccles M.  The potential influence of small group processes on guideline development. J Eval Clin Pract. 2001; 7.165-73
PubMed
 
Popay J, Williams G.  Qualitative research and evidence-based healthcare. J R Soc Med. 1998; 91.Suppl 3532-7
PubMed
 
Pronovost PJ, Berenholtz SM, Needham DM.  Translating evidence into practice: a model for large scale knowledge translation. BMJ. 2008; 337.a1714
PubMed
 
Raine R, Sanderson C, Black N.  Developing clinical guidelines: a challenge to current methods. BMJ. 2005; 331.631-3
PubMed
 
Rycroft-Malone J.  Formal consensus: the development of a national clinical guideline. Qual Health Care. 2001; 10.238-44
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD, WHO Advisory Committee on Health Research.  Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst.. 2006; 4.13
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation. Health Res Policy Syst. 2006; 4.25
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations. Health Res Policy Syst.. 2006; 4.21
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement. Health Res Policy Syst. 2006; 4.22
PubMed
 
Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M.  Are nonspecific practice guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000; 34.1429-48
PubMed
 
Shekelle PG, Schriger DL.  Evaluating the use of the appropriateness method in the Agency for Health Care Policy and Research Clinical Practice Guideline Development process. Health Serv Res. 1996; 31.453-68
PubMed
 
McGory ML, Shekelle PG, Rubenstein LZ, Fink A, Ko CY.  Developing quality indicators for elderly patients undergoing abdominal operations. J Am Coll Surg. 2005; 201.870-83
PubMed
 
Shekelle PG, Woolf SH, Eccles M, Grimshaw J.  Clinical guidelines: developing guidelines. BMJ. 1999; 318.593-6
PubMed
 
Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G. et al.  The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005; 5.23
PubMed
 
Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM.  Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003; 139.493-8
PubMed
 
AGREE Collaboration.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003; 12.18-23
PubMed
 
Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr.  Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009; 301.831-41
PubMed
 
Van der Wees PJ, Moore AP, Powers CM, Stewart A, Nijhuis-van der Sanden MW, de Bie RA.  Development of clinical guidelines in physical therapy: perspective for international collaboration. Phys Ther. 2011; 91.1551-63
PubMed
 
Winn RJ, McClure J.  The NCCN clinical practice guidelines in oncology: a primer for users. J Natl Compr Canc Netw. 2003; 1.5-13
PubMed
 
Woolf SH.  Practice guidelines, a new reality in medicine. II. Methods of developing guidelines. Arch Intern Med. 1992; 152.946-52
PubMed
 
Woolf SH, DiGuiseppi CG, Atkins D, Kamerow DB.  Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health. 1996; 17.511-38
PubMed
 
Francke AL, Smit MC, de Veer AJ, Mistiaen P.  Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008; 8.38
PubMed
 
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L. et al.  Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004; 8.iii-iv, 1-72
PubMed
 
Gross PA, Greenfield S, Cretin S, Ferguson J, Grimshaw J, Grol R. et al.  Optimal methods for guideline implementation: conclusions from Leeds Castle meeting. Med Care. 2001; 39.II85-92
PubMed
 
Rubenstein LV, Pugh J.  Strategies for promoting organizational and practice change by advancing implementation research. J Gen Intern Med. 2006; 21.Suppl 2S58-64
PubMed
 
Grupo de trabajo sobre GPC.  Elaboración de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. Guías de Práctica Clínica en el SNS and Instituto Aragonés de Ciencias de la Salud no. 2006/0I. Madrid: Ministerio de Sanidad y Consumo and Instituto Aragonés de Ciencias de la Salud; 2007.
 
Wedzicha W, Fletcher M, Powell P.  Making ERS guidelines relevant and accessible: involving patients and the public. Breathe. 2011; 8.9-11
 
Institute of Medicine.  Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Pr; 2009.
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153.194-9
PubMed
 
Grol R, Grimshaw J.  From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003; 362.1225-30
PubMed
 
Cook DJ, Mulrow CD, Haynes RB.  Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997; 126.376-80
PubMed
 
Agency for Healthcare Research and Quality National Guideline Clearinghouse.  Accessed at www.guideline.gov/ on 9 August 2011.
 
Lomotan EA, Michel G, Lin Z, Shiffman RN.  How “should” we write guideline recommendations? Interpretation of deontic terminology in clinical practice guidelines: survey of the health services community. Qual Saf Health Care. 2010; 19.509-13
PubMed
 
Appendix: G-I-N Board of Trustees

Jako Burgers (the Netherlands), Dave Davis (United States), Frode Forland (Norway), Minna Kaila (Finland), Fergus Macbeth (United Kingdom), Günter Ollenschläger (Germany), Sue Phillips (Australia), Keng Ho Pwee (Singapore), Amir Qaseem (United States), Rosa Rico (Spain), Jean Slutsky (United States), Sara Twaddle (United Kingdom), and Philip van der Wees (Netherlands).

Figures

Tables

Table Jump Placeholder

Table. Key Components of High-Quality and Trustworthy Guidelines

References

Qaseem A, Snow V, Gosfield A, Gregg D, Michl K, Wennberg D. et al.  Pay for performance through the lens of medical professionalism. Ann Intern Med. 2010; 152.366-9
PubMed
 
Institute of Medicine.  Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Pr; 2011.
 
Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM.  Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003; 139.493-8
PubMed
 
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE Next Steps Consortium.  Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010; 182.472-8
PubMed
CrossRef
 
World Health Organization.  Global Programme on Evidence for Health Policy. Geneva: World Health Organization; 2003: Accessed at http://whqlibdoc.who.int/hq/2003/EIP_GPE_EQC_2003_1.pdf on 1 October 2011.
 
The National Institute for Health and Clinical Excellence.  The Guidelines Manual (January 2009). London; 2009. Accessed at www.nice.org.uk/guidelinesmanual on 1 October 2011.
 
Scottish Intercollegiate Guidelines Network.  SIGN 50: A Guideline Developer's Handbook. Edinburgh, Scotland: Scottish Intercollegiate Guidelines Network; 2008. Accessed at www.sign.ac.uk/guidelines/fulltext/50/index.html on 1 October 2011.
 
National Health and Medical Research Council.  Procedures and Requirements for Meeting the 2011 NHMRC Standard for Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2011.
 
American Society of Clinical Oncology.  American Society of Clinical Oncology Guideline Procedures Manual. Expert Panel Version 4.0. Alexandria, VA: American Soc Clinical Oncology; 2011. Accessed at www.asco.org/ASCO/Downloads/Cancer%20Policy%20and%20Clinical%20Affairs/Clinical%20Affairs%20(derivative%20products)/Manual/Methodology%20Manual%201.25.11.pdf on 1 October 2011.
 
American College of Cardiology Foundation and American Heart Association.  Methodology Manual and Policies from the ACCF/AHA Task Force on Practice Guidelines. 2010. Accessed at http://my.americanheart.org/professional/StatementsGuidelines/PoliciesDevelopment/Development/Methodologies-and-Policies-from-the-ACCAHA-Task-Force-on-Practice-Guidelines_UCM_320470_Article.jsp on 1 October 2011.
 
Baumann MH, Lewis SZ, Gutterman D, American College of Chest Physicians.  ACCP evidence-based guideline development: a successful and transparent approach addressing conflict of interest, funding, and patient-centered recommendations. Chest. 2007; 132.1015-24
PubMed
 
Edlund W, Gronseth G, Yuen S, Franklin G; Quality Standards Subcommittee and Therapeutics and Technology Assessment Subcommittee.  American Academy of Neurology Clinical Practice Guideline Process Manual. 2004. Accessed at www.aan.com/globals/axon/assets/3749.pdf on 1 October 2011.
 
 Attributes of ATS documents that guide clinical practice. Recommendations of the ATS Clinical Practice Committee. Am J Respir Crit Care Med. 1997; 156.2015-25
PubMed
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153.194-9
PubMed
 
The Intensive Care Society.  SSQ Committee Guideline Policy. 2010. Accessed at www.ics.ac.uk/intensive_care_professional/standards_and_guidelines/ssq_guideline_policy on 1 October 2011.
 
Committee of Ministers of the Council of Europe.  Developing a Metholology for Drawing up Guidelines on Best Medical Practices: Recommendation Rec(2001)13. Strasbourg, Cedex, France: Council of Europe Publishing; 2001. Accessed at www.leitlinien.de/mdb/edocs/pdf/literatur/coe-rec-2001-13.pdf on 1 October 2011.
 
Dutch Institute for Healthcare Improvement CBO.  Accessed at www.cbo.nl/en/ on 1 October 2011.
 
German Agency for Quality in Medicine.  Accessed at www.aezq.de/ on 1 October 2011.
 
Grol R, Cluzeau FA, Burgers JS.  Clinical practice guidelines: towards better quality guidelines and increased international collaboration [Editorial]. Br J Cancer. 2003; 89.Suppl 1S4-8
PubMed
 
Guidelines International Network.  Development and Training Resources. Accessed at www.g-i-n.net/library/development-and-training-resources on 1 October 2011.
 
McDonald CJ, Overhage JM.  Guidelines you can follow and can trust. An ideal and an example [Editorial]. JAMA. 1994; 271.872-3
PubMed
 
Mlika-Cabanne N, Harbour R, de Beer H, Laurence M, Cook R, Twaddle S, Guidelines International Network (GIN) Working Group on Evidence Tables.  Sharing hard labour: developing a standard template for data summaries in guideline development. BMJ Qual Saf. 2011; 20.141-5
PubMed
 
Rosenfeld RM, Shiffman RN.  Clinical practice guidelines: a manual for developing evidence-based guidelines to facilitate performance measurement and quality improvement. Otolaryngol Head Neck Surg. 2006; 135.S1-28
PubMed
 
Rosenfeld RM, Shiffman RN.  Clinical practice guideline development manual: a quality-driven approach for translating evidence into action. Otolaryngol Head Neck Surg. 2009; 140.S1-43
PubMed
 
Institute of Medicine.  Finding What Works in Healthcare. Washington, DC: National Academies Pr; 2011.
 
Kuehn BM.  IOM sets out “gold standard” practices for creating guidelines, systematic reviews. JAMA. 2011; 305.1846-8
PubMed
 
Ollenschläger G, Marshall C, Qureshi S, Rosenbrand K, Burgers J, Mäkelä M, et al. Board of Trustees 2002, Guidelines International Network (G-I-N).  Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004; 13.455-60
PubMed
 
Grilli R, Magrini N, Penna A, Mura G, Liberati A.  Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000; 355.103-6
PubMed
 
Shaneyfelt TM, Centor RM.  Reassessment of clinical practice guidelines: go gently into that good night [Editorial]. JAMA. 2009; 301.868-9
PubMed
 
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J.  Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999; 281.1900-5
PubMed
 
AGREE Collaboration.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003; 12.18-23
PubMed
 
Ansari MT, Tsertsvadze A, Moher D.  Grading quality of evidence and strength of recommendations: a perspective. PLoS Med. 2009; 6.1000151
PubMed
 
Appraisal of Guidelines for Research and Evaluation Instrument.  2001. Accessed at www.agreetrust.org/ on 1 October 2011.
 
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004; 328.1490
PubMed
 
Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al. GRADE Working Group.  Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005; 5.25
PubMed
 
Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. GRADE Working Group.  Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004; 4.38
PubMed
 
Browman GP.  Improving clinical practice guidelines for the 21st century. Attitudinal barriers and not technology are the main challenges. Int J Technol Assess Health Care. 2000; 16.959-68
PubMed
 
Browman GP.  Evidence-based clinical practice guideline development: principles, challenges, and accountability to evidence [Editorial]. J Surg Oncol. 2010; 101.1-2
PubMed
 
Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J, AGREE Collaboration.  Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003; 15.31-45
PubMed
 
Burgers JS, Grol RP, Zaat JO, Spies TH, van der Bij AK, Mokkink HG.  Characteristics of effective clinical guidelines for general practice. Br J Gen Pract. 2003; 53.15-9
PubMed
 
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA. et al.  Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999; 282.1458-65
PubMed
 
Calderón C, Rotaeche R, Etxebarria A, Marzo M, Rico R, Barandiaran M.  Gaining insight into the Clinical Practice Guideline development processes: qualitative study in a workshop to implement the GRADE proposal in Spain. BMC Health Serv Res. 2006; 6.138
PubMed
 
Calogne N, Harris R.  United States Preventive Services Task Force (USPSTF). Presented at Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Irvine, California, 19 April 2010.
 
Carter AO, Battista RN, Hodge MJ, Lewis S, Basinski A, Davis D.  Report on activities and attitudes of organizations active in the clinical practice guidelines field. CMAJ. 1995; 153.901-7
PubMed
 
Christiaens T, De Backer D, Burgers J, Baerheim A.  Guidelines, evidence, and cultural factors. Scand J Prim Health Care. 2004; 22.141-5
PubMed
 
Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR.  The relation between systematic reviews and practice guidelines. Ann Intern Med. 1997; 127.210-6
PubMed
 
Davis D, Goldman J, Palda V.  Handbook on Clinical Practice Guidelines. Toronto, Canada: Canadian Medical Assoc; 2007.
 
Eccles M, Freemantle N, Mason J, Using systematic reviews in clinical guideline development.Ross AC, Taylor CL, Yaktine AL, Del Valle HB.  Systematic Reviews in Health Care: Meta-Analysis in Context. 2nd ed. Systematic Reviews in Health Care. London: BMJ Publishing Group; 2008.400
 
Fervers B, Burgers JS, Haugh MC, Brouwers M, Browman G, Cluzeau F. et al.  Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care. 2005; 17.123-32
PubMed
 
Fink A, Kosecoff J, Chassin M, Brook RH.  Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984; 74.979-83
PubMed
 
Freeman AC, Sweeney K.  Why general practitioners do not implement evidence: qualitative study. BMJ. 2001; 323.1100-2
PubMed
 
Fretheim A, Schünemann HJ, Oxman AD.  Improving the use of research evidence in guideline development: 5. Group processes. Health Res Policy Syst. 2006; 4.17
PubMed
 
Fretheim A, Schünemann HJ, Oxman AD.  Improving the use of research evidence in guideline development: 3. Group composition and consultation process. Health Res Policy Syst. 2006; 4.15
PubMed
 
Gabbay J, le May A.  Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care. BMJ. 2004; 329.1013
PubMed
 
Garfield FB, Garfield JM.  Clinical judgment and clinical practice guidelines. Int J Technol Assess Health Care. 2000; 16.1050-60
PubMed
 
Giacomini MK.  The rocky road: qualitative research as evidence. Evid Based Med. 2001; 6.4-6
 
Graham ID, Harrison MB.  Evaluation and adaptation of clinical practice guidelines. Evid Based Nurs. 2005; 8.68-72
PubMed
 
Green J, Britten N.  Qualitative research and evidence based medicine. BMJ. 1998; 316.1230-2
PubMed
 
Greenhalgh T.  “Is my practice evidence-based?” [Editorial]. BMJ. 1996; 313.957-8
PubMed
 
Griffiths F, Green E, Tsouroufli M.  The nature of medical evidence and its inherent uncertainty for the clinical consultation: qualitative study. BMJ. 2005; 330.511
PubMed
 
Grimshaw J, Eccles M, Russell I.  Developing clinically valid practice guidelines. J Eval Clin Pract. 1995; 1.37-48
PubMed
 
Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I. et al.  Developing and implementing clinical practice guidelines. Qual Health Care. 1995; 4.55-64
PubMed
 
Grimshaw JM, Russell IT.  Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993; 342.1317-22
PubMed
 
Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H.  Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998; 317.858-61
PubMed
 
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. GRADE Working Group.  Grading quality of evidence and strength of recommendations. BMJ. 2004; 328.1490
PubMed
 
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE Working Group.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336.924-6
PubMed
 
Haines A, Jones R.  Implementing findings of research. BMJ. 1994; 308.1488-92
PubMed
 
Harris JS, Sinnott PL, Holland JP, Ording J, Turkelson C, Weiss M. et al.  Methodology to update the practice recommendations in the American College of Occupational and Environmental Medicine's Occupational Medicine Practice Guidelines, second edition. J Occup Environ Med. 2008; 50.282-95
PubMed
 
Harrison MB, Légaré F, Graham ID, Fervers B.  Adapting clinical practice guidelines to local context and assessing barriers to their use. CMAJ. 2010; 182.78-84
PubMed
 
Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB.  More informative abstracts of articles describing clinical practice guidelines. Ann Intern Med. 1993; 118.731-7
PubMed
 
Heathfield H, Pitty D, Hanka R.  Evaluating information technology in health care: barriers and challenges. BMJ. 1998; 316.1959-61
PubMed
 
Hutchings A, Raine R.  A systematic review of factors affecting the judgments produced by formal consensus development methods in health care. J Health Serv Res Policy. 2006; 11.172-9
PubMed
 
Institute of Medicine.  Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Pr; 2009.
 
Jaime J.  Introduction: Practice Guidelines. Helpful Aids or Paradigm Shift? Guideline Development Methods: Information for National Collaborating Centres and Guideline Developers. Int J Technol Assess Health Care. 2005; 16.957-8
 
Kahan JP, Park RE, Leape LL, Bernstein SJ, Hilborne LH, Parker L. et al.  Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures. Med Care. 1996; 34.512-23
PubMed
 
Kavanagh BP.  The GRADE system for rating clinical guidelines. PLoS Med. 2009; 6.1000094
PubMed
 
Krahn M, Naglie G.  The next step in guideline development: incorporating patient preferences. JAMA. 2008; 300.436-8
PubMed
 
Lewis S.  Paradox, process and perception: the role of organizations in clinical practice guidelines development. CMAJ. 1995; 153.1073-7
PubMed
 
Moreira T.  Diversity in clinical guidelines: the role of repertoires of evaluation. Soc Sci Med. 2005; 60.1975-85
PubMed
 
Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P.  Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess. 1998; 2.iii-ix, 1-274
PubMed
 
National Health and Medical Research Council.  A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 1999.
 
National Health and Medical Research Council.  NHMRC Levels of Evidence and Grades for Recommendations for Developers of Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2009.
 
www.nhmrc.gov.au/guidelines/procedures-and-requirements-meeting-2011-nhmrc-standard-clinical-practice-guidelinesNational Health and Medical Research Council.  Procedures and Requirements for Meeting the 2011 NHMRC Standard for Clinical Practice Guidelines. Melbourne, Australia: National Health and Medical Research Council; 2011.
 
Owens DK.  Improving practice guidelines with patient-specific recommendations [Editorial]. Ann Intern Med. 2011; 154.638-9
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst. 2006; 4.20
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 7. Deciding what evidence to include. Health Res Policy Syst. 2006; 4.19
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A.  Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst.. 2006; 4.14
PubMed
 
Oxman AD, Schünemann HJ, Fretheim A, Subcommittee on the Use of Research Evidence.  Improving the use of research evidence in guideline development. Report no. 11-2007. Norwegian Knowledge Centre for the Health Services; 2007.
 
Pagliari C, Grimshaw J.  Impact of group structure and process on multidisciplinary evidence-based guideline development: an observational study. J Eval Clin Pract. 2002; 8.145-53
PubMed
 
Pagliari C, Grimshaw J, Eccles M.  The potential influence of small group processes on guideline development. J Eval Clin Pract. 2001; 7.165-73
PubMed
 
Popay J, Williams G.  Qualitative research and evidence-based healthcare. J R Soc Med. 1998; 91.Suppl 3532-7
PubMed
 
Pronovost PJ, Berenholtz SM, Needham DM.  Translating evidence into practice: a model for large scale knowledge translation. BMJ. 2008; 337.a1714
PubMed
 
Raine R, Sanderson C, Black N.  Developing clinical guidelines: a challenge to current methods. BMJ. 2005; 331.631-3
PubMed
 
Rycroft-Malone J.  Formal consensus: the development of a national clinical guideline. Qual Health Care. 2001; 10.238-44
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD, WHO Advisory Committee on Health Research.  Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst.. 2006; 4.13
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation. Health Res Policy Syst. 2006; 4.25
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations. Health Res Policy Syst.. 2006; 4.21
PubMed
 
Schünemann HJ, Fretheim A, Oxman AD.  Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement. Health Res Policy Syst. 2006; 4.22
PubMed
 
Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M.  Are nonspecific practice guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000; 34.1429-48
PubMed
 
Shekelle PG, Schriger DL.  Evaluating the use of the appropriateness method in the Agency for Health Care Policy and Research Clinical Practice Guideline Development process. Health Serv Res. 1996; 31.453-68
PubMed
 
McGory ML, Shekelle PG, Rubenstein LZ, Fink A, Ko CY.  Developing quality indicators for elderly patients undergoing abdominal operations. J Am Coll Surg. 2005; 201.870-83
PubMed
 
Shekelle PG, Woolf SH, Eccles M, Grimshaw J.  Clinical guidelines: developing guidelines. BMJ. 1999; 318.593-6
PubMed
 
Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G. et al.  The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005; 5.23
PubMed
 
Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM.  Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003; 139.493-8
PubMed
 
AGREE Collaboration.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003; 12.18-23
PubMed
 
Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr.  Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009; 301.831-41
PubMed
 
Van der Wees PJ, Moore AP, Powers CM, Stewart A, Nijhuis-van der Sanden MW, de Bie RA.  Development of clinical guidelines in physical therapy: perspective for international collaboration. Phys Ther. 2011; 91.1551-63
PubMed
 
Winn RJ, McClure J.  The NCCN clinical practice guidelines in oncology: a primer for users. J Natl Compr Canc Netw. 2003; 1.5-13
PubMed
 
Woolf SH.  Practice guidelines, a new reality in medicine. II. Methods of developing guidelines. Arch Intern Med. 1992; 152.946-52
PubMed
 
Woolf SH, DiGuiseppi CG, Atkins D, Kamerow DB.  Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health. 1996; 17.511-38
PubMed
 
Francke AL, Smit MC, de Veer AJ, Mistiaen P.  Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008; 8.38
PubMed
 
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L. et al.  Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004; 8.iii-iv, 1-72
PubMed
 
Gross PA, Greenfield S, Cretin S, Ferguson J, Grimshaw J, Grol R. et al.  Optimal methods for guideline implementation: conclusions from Leeds Castle meeting. Med Care. 2001; 39.II85-92
PubMed
 
Rubenstein LV, Pugh J.  Strategies for promoting organizational and practice change by advancing implementation research. J Gen Intern Med. 2006; 21.Suppl 2S58-64
PubMed
 
Grupo de trabajo sobre GPC.  Elaboración de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. Guías de Práctica Clínica en el SNS and Instituto Aragonés de Ciencias de la Salud no. 2006/0I. Madrid: Ministerio de Sanidad y Consumo and Instituto Aragonés de Ciencias de la Salud; 2007.
 
Wedzicha W, Fletcher M, Powell P.  Making ERS guidelines relevant and accessible: involving patients and the public. Breathe. 2011; 8.9-11
 
Institute of Medicine.  Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Pr; 2009.
 
Qaseem A, Snow V, Owens DK, Shekelle P, Clinical Guidelines Committee of the American College of Physicians.  The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods. Ann Intern Med. 2010; 153.194-9
PubMed
 
Grol R, Grimshaw J.  From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003; 362.1225-30
PubMed
 
Cook DJ, Mulrow CD, Haynes RB.  Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997; 126.376-80
PubMed
 
Agency for Healthcare Research and Quality National Guideline Clearinghouse.  Accessed at www.guideline.gov/ on 9 August 2011.
 
Lomotan EA, Michel G, Lin Z, Shiffman RN.  How “should” we write guideline recommendations? Interpretation of deontic terminology in clinical practice guidelines: survey of the health services community. Qual Saf Health Care. 2010; 19.509-13
PubMed
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Journal Club
Topic Collections
PubMed Articles

Want to Subscribe?

Learn more about subscription options

Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)