Evaluating a Practice Protocol Take this opportunity to examine clinical guidelines or practice protocols from your specialty are
Assignment:
Evaluating a Practice Protocol
Take this opportunity to examine clinical guidelines or practice protocols from your specialty area.
Do the following assignments:
Complete the protocol evaluation activity by following these instructions:
- Select a practice protocol related to your clinical setting. These may be from your agency’s policy and procedure manual used in your practice.
- Determine who wrote the protocol and request an interview with one or more individuals who were involved in developing the protocol. Interview questions may include, but are not limited to the following:
- Can you explain how protocols are developed in your agency?
- What specific information or data was used to develop this protocol?
- Are there any references you can identify that would give me a better understanding of the rationale for the practices used in the protocol?
- What rating system was used to critique the protocol guidelines?
- After the interview, categorize the sources of knowledge identified by the interviewee (e.g., authority, experience, research-based). Be specific.
- After the interview, you may not have enough information on which to rate the protocol. If, for example, the interviewee has not provided you with sources from the literature, you will have to do a search to determine whether research exists to support the protocol you are assessing. This does not have to be an exhaustive review, yet you should be able to determine whether there is valid research there to support your rating (see next step).
- Then, using the AGREE ll rating scale (see attached) , assess the protocol for its validity in application to practice.
- Complete a Brief Evaluation of the Protocol Outline by following these steps:
- Brief description of the protocol.
- Title and credentials of the individuals who developed the protocol.
- Parameters of the literature search and a summary of how the protocol was developed.
- Your summary conclusion about the types and quality of knowledge upon which the protocol was based with supporting rationale.
- A brief description of the AGREE rating scale used to assess the protocol.
- Your assessment of the validity of the selected protocol.
- Upload the following in the Assignment Dropbox:
- the Brief Evaluation (required elements listed below);
- the completed AGREE ll scale; and
- a copy of the clinical protocol in the designated Assignment Drop Box for this Module
APPRAISAL OF GUIDELINES fOR RESEARCH & EVALUATION II
INSTRUMENT
___________________________________________________________________________________
The AGREE Next Steps Consortium May 2009
UPDATE: September 2013
ii
COPYRIGHT AND REPRODUCTION
This document is the product of an international collaboration. It may be reproduced and used for educational purposes,
quality assurance programmes and critical appraisal of guidelines. It may not be used for commercial
purposes or product marketing. Approved non-English language versions of the AGREE II Instrument must be used where available. Offers of assistance in translation into other
languages are welcome, provided they conform to the protocol set out by The AGREE Research Trust.
DISCLAIMER
The AGREE II Instrument is a generic tool designed primarily to help guideline developers and users assess the
methodological quality of guidelines. The authors do not take responsibility for the improper use of
the AGREE II Instrument.
© The AGREE Research Trust, May 2009. © The AGREE Research Trust, September 2013.
SUGGESTED CITATION: Brouwers M, Kho ME, Browman GP, Cluzeau F, feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can
Med Assoc J. Dec 2010, 182:E839-842; doi: 10.1503/cmaj.090449
FUNDING: The development of the AGREE II Instrument was funded by a
grant from the Canadian Institutes of Health Research (FRN77822)
FOR FURTHER INFORMATION ABOUT THE AGREE CONTACT:
AGREE II Project Office, [email protected] AGREE Research Trust Web site, www.agreetrust.org
AGREE 10th Year Anniversary: 2003 -‐ 2013
iii
AGREE NEXT STEPS CONSORTIUM MEMBERSHIP Dr. Melissa C. Brouwers Principal Investigator, AGREE Next Steps Consortium McMaster University, Hamilton, Ontario, Canada Consortium Members: Dr. GP. Browman, British Columbia Cancer Agency, Vancouver Island, Canada Dr. JS. Burgers, Dutch Institute for Healthcare Improvement CBO, The Netherlands Dr. F. Cluzeau, Chair of AGREE Research Trust; St. George’s Hospital Medical School, London, UK Dr. D. Davis, Association of American Medical Colleges, Washington, DC, USA Dr. G. Feder, University of Bristol, UK Dr. B. Fervers, Cancer et Environement, Centre Léon Bérard, France Dr. I. Graham, Canadian Institutes of Health Research, Ottawa, Ontario, Canada Dr. J. Grimshaw, Ottawa Health Research Institute, Ontario, Canada Dr. SE. Hanna, McMaster University, Hamilton, Ontario, Canada Ms. ME. Kho, McMaster University, Hamilton, Ontario Canada Dr. P. Littlejohns, National Institute for Health and Clinical Excellence, London, UK Ms. J. Makarski, McMaster University, Hamilton, Ontario, Canada Dr. L. Zitzelsberger, Canadian Partnership Against Cancer, Ottawa, Ontario, Canada
NOTICE: AGREE II Original Public Release and Publication Date: 2009/2010
AGREE II Update: September 2013 Please see the Update section, at the end of the Introduction, following the References section
iv
TABLE OF CONTENTS
I. INTRODUCTION
I. Overview ……………………………………………………………………………………………………………………. 1
II. Applying the AGREE II ………………………………………………………………………………………………….. 4
III. Key Resources and References ……………………………………………………………………………………… 4
II. USER’S MANUAL: INSTRUCTIONS FOR USING THE AGREE II ……………………………………………………….. 6
I. Preparing to Use the AGREE II ………………………………………………………………………………………. 7
II. Structure and Content of the AGREE II …………………………………………………………………………… 7
III. Rating Scale and User’s Manual Sections ………………………………………………………………………… 8
IV. Scoring the AGREE II …………………………………………………………………………………………………….. 9
V. Overall Assessment ……………………………………………………………………………………………………. 10
VI. Guidance for Rating Each Item
a. Domain 1. Scope and Purpose ……………………………………………………………………………….. 11
b. Domain 2. Stakeholder Involvement ……………………………………………………………………….. 15
c. Domain 3. Rigour of Development …………………………………………………………………………. 19
d. Domain 4. Clarity of Presentation ………………………………………………………………………….. 28
e. Domain 5. Applicability …………………………………………………………………………………………. 32
f. Domain 6. Editorial Independence ………………………………………………………………………….. 37
g. Overall Guideline Assessment ………………………………………………………………………………… 40
III. AGREE II INSTRUMENT …………………………………………………………………………………………………………… 1
I. Domain 1. Scope and Purpose ………………………………………………………………………………………. 2
II. Domain 2. Stakeholder Involvement ……………………………………………………………………………… 3
III. Domain 3. Rigour of Development ………………………………………………………………………………… 4
IV. Domain 4. Clarity of Presentation ………………………………………………………………………………….. 7
V. Domain 5. Applicability ………………………………………………………………………………………………… 8
VI. Domain 6. Editorial Independence ………………………………………………………………………………. 10
VII. Overall Guideline Assessment ……………………………………………………………………………………… 11
1
I. INTRODUCTION
I. OVERVIEW
i) Purpose of the AGREE II Instrument Clinical practice guidelines (‘guidelines’) are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (1). In addition, guidelines can play an important role in health policy formation (2,3) and have evolved to cover topics across the health care continuum (e.g., health promotion, screening, diagnosis). The potential benefits of guidelines are only as good as the quality of the guidelines themselves. Appropriate methodologies and rigorous strategies in the guideline development process are important for the successful implementation of the resulting recommendations (4-6). The quality of guidelines can be extremely variable and some often fall short of basic standards (7-9). The Appraisal of Guidelines for REsearch & Evaluation (AGREE) Instrument (10) was developed to address the issue of variability in guideline quality. To that end, the AGREE instrument is a tool that assesses the methodological rigour and transparency in which a guideline is developed. The original AGREE instrument has been refined, which has resulted in the new AGREE II and includes a new User’s Manual (11). The purpose of the AGREE II, is to provide a framework to:
1. assess the quality of guidelines; 2. provide a methodological strategy for the development of guidelines; and 3. inform what information and how information ought to be reported in guidelines. The AGREE II replaces the original instrument as the preferred tool and can be used as part of an overall quality mandate aimed to improve health care.
ii) History of the AGREE Project The original AGREE Instrument was published in 2003 by a group of international guideline developers and researchers, the AGREE Collaboration (10). The objective of the Collaboration was to develop a tool to assess the quality of guidelines. The AGREE Collaboration defined quality of guidelines as the confidence that the potential biases of guideline development have been addressed adequately and that the recommendations are both internally and externally valid, and are feasible for practice (10). The assessment includes judgments about the methods used for developing the guidelines, the components of the final recommendations, and the factors that are linked to their uptake. The result of the Collaboration’s effort was the original AGREE Instrument, a 23-item tool comprising 6 quality domains. The AGREE Instrument has been translated into many languages, has been cited in well over 100 publications, and is endorsed by several health care organizations. More details about the original instrument and related publications are available on the Web site of the AGREE Research Trust (http://www.agreetrust.org/), the official body managing the interests of the AGREE Instrument.
2
As with any new assessment tool, it was recognized that ongoing development was required to strengthen the measurement properties of the instrument and to ensure its usability and feasibility among intended users. This led several members of the original team to form the AGREE Next Steps Consortium (Consortium). The objectives of the Consortium were to further improve the measurement properties of the instrument, including its reliability and validity; to refine the instrument’s items to better meet the needs of the intended users; and to improve the supporting documentation (i.e., original training manual and user’s guide) to facilitate the ability of users to implement the instrument with confidence. The result of these efforts is the AGREE II, which is comprised of the new User’s Manual and 23 item tool organized into the same six domains, described here. The User’s Manual is a significant modification of the original training manual and user’s guide and provides explicit information for each of the 23 items. Table 1 compares the items of the original AGREE to the items in the AGREE II. Table 1. Comparison of original AGREE and AGREE II items.
Original AGREE Item AGREE II Item
Domain 1. Scope and Purpose
1. The overall objective(s) of the guideline is (are) specifically described.
No change
2. The clinical question(s) covered by the guideline is (are) specifically described.
The health question(s) covered by the guideline is (are) specifically described.
3. The patients to whom the guideline is meant to apply are specifically described.
The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.
Domain 2. Stakeholder Involvement
4. The guideline development group includes individuals from all the relevant professional groups.
No change
5. The patients’ views and preferences have been sought. The views and preferences of the target population (patients, public, etc.) have been sought.
6. The target users of the guideline are clearly defined. No change
7. The guideline has been piloted among end users. Delete item. Incorporated into user guide description of item 19.
Domain 3. Rigour of Development
8. Systematic methods were used to search for evidence. No change in item. Renumber to 7.
9. The criteria for selecting the evidence are clearly described.
No change in item. Renumber to 8.
NEW Item 9. The strengths and limitations of the body of evidence are clearly described.
10. The methods for formulating the recommendations are clearly described.
No change
11. The health benefits, side effects, and risks have been considered in formulating the recommendations.
No change
3
Original AGREE Item AGREE II Item
12. There is an explicit link between the recommendations and the supporting evidence.
No change
13. The guideline has been externally reviewed by experts prior to its publication.
No change
14. A procedure for updating the guideline is provided. No change
Domain 4. Clarity of Presentation
15. The recommendations are specific and unambiguous. No change
16. The different options for management of the condition are clearly presented.
The different options for management of the condition or health issue are clearly presented.
17. Key recommendations are easily identifiable. No change
Domain 5. Applicability
18. The guideline is supported with tools for application. The guideline provides advice and/or tools on how the recommendations can be put into practice.
AND Change in domain (from Clarity of Presentation) AND renumber to 19
19. The potential organizational barriers in applying the recommendations have been discussed.
The guideline describes facilitators and barriers to its application.
AND change in order – renumber to 18
20. The potential cost implications of applying the recommendations have been considered.
The potential resource implications of applying the recommendations have been considered.
21. The guideline presents key review criteria for monitoring and/ or audit purposes.
The guideline presents monitoring and/ or auditing criteria.
Domain 6. Editorial Independence
22. The guideline is editorially independent from the funding body.
The views of the funding body have not influenced the content of the guideline.
23. Conflicts of interest of guideline development members have been recorded.
Competing interests of guideline development group members have been recorded and addressed.
4
II. APPLYING THE AGREE II
i) Which guidelines can be appraised with the AGREE II? As with the original instrument, AGREE II is designed to assess guidelines developed by local, regional, national or international groups or affiliated governmental organizations. These include original versions of and updates of existing guidelines. The AGREE II is generic and can be applied to guidelines in any disease area targeting any step in the health care continuum, including those for health promotion, public health, screening, diagnosis, treatment or interventions. It is suitable for guidelines presented in paper or electronic format. At this stage, the AGREE II has not been designed to assess the quality of guidance documents that address health care organizational issues. Its role in the assessment of health technology assessments has not yet been formally evaluated.
ii) Who can use the AGREE II? The AGREE II is intended to be used by the following stakeholder groups:
• by health care providers who wish to undertake their own assessment of a guideline before adopting its recommendations into their practice;
• by guideline developers to follow a structured and rigorous development methodology, to conduct an internal assessment to ensure that their guidelines are sound, or to evaluate guidelines from other groups for potential adaptation to their own context;
• by policy makers to help them decide which guidelines could be recommended for use in practice or to inform policy decisions; and
• by educators to help enhance critical appraisal skills amongst health professionals and to teach core competencies in guideline development and reporting.
III. KEY RESOURCES AND REFERENCES
i) AGREE Research Trust The AGREE Research Trust (ART) is an independent body established in 2004 at the conclusion of the activities of the original AGREE Collaboration. ART endorses the AGREE II and manages the interests of the AGREE enterprise, supports a research agenda regarding its development, and serves as the holder of its copyright. The AGREE Research Trust web site http://www.agreetrust.org provides: • free downloadable copies of AGREE II • links to the AGREE II on-line training tool • reference lists citing AGREE II and the original AGREE Instrument • free downloadable copies of the original AGREE Instrument • information about AGREE projects, the AGREE Next Steps Consortium and the original
AGREE Collaboration
5
ii) How to cite the AGREE II AGREE Next Steps Consortium (2009). The AGREE II Instrument [Electronic version]. Retrieved <Month, Day, Year>, from http://www.agreetrust.org . iii) AGREE II On-Line Training Tool For access to the AGREE II On-Line Training Tool, please visit http://www.agreetrust.org . iv) References related to the AGREE II AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Parallel publications in progress v) Primary reference related to the original AGREE Instrument AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003 Feb;12(1):18-23. REFERENCES 1. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential
benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527-530. 2. Committee to Advise the Public Health Service on Clinical Practice Guidelines IoM. Clinical
practice guidelines: directions for a new program. Washington: National Academy Press; 1990.
3. Browman GP, Snider A, Ellis P. Negotiating for change. The healthcare manager as catalyst for evidence-based practice: changing the healthcare environment and sharing experience. Healthc Pap. 2003;3(3):10-22.
4. Grol R. Success and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001;39(8 Suppl 2):1146-54.
5. Davis DA, Taylor-Vaisey A. Translating guidelines into practice: a systematic review of theoretic concepts, practice experience and research evidence in the adoption of clinical practice guidelines. CMAJ. 1997;157(4):408-16.
6. Grimshaw J,.Russell I. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317-22.
7. Shaneyfelt TM, Mayo-Smith MF Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA 1999:281(20):1900-5.
8. Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for critical appraisal. Lancet. 2000;355:103-6.
9. Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Phillip T, Cluzeau FA. International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. J Clin Oncol. 2004;22:2000-7.
10. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003 Feb;12(1):18-23.
11. AGREE II: Advancing the guideline development, reporting and evaluation in healthcare. Parallel publications in progress.
UPDATE: September 2013 In 2013, the AGREE marked its 10th anniversary since the original AGREE Instrument was first published and made available for use. To mark this anniversary, we provide a summary of
6
activities that have taken place over the past 10 years and an update to the references originally noted in the AGREE II 2009 version. UPTAKE AND USE OF AGREE II As with the original AGREE Instrument, uptake and use of AGREE II has been significant. Between 2010 (publication of AGREE II) and July 2013, a cited reference search revealed a total of 590 articles referencing the four core AGREE articles. An analysis of the AGREE Enterprise Website (www.agreetrust.org) showed much activity with a total of 42,553 visits to the website over a seven month period (January – July 2013). In addition, to date nearly 2,000 users have registered accounts with the website. AGREE II continues to be used as the basis of PG development frameworks, academic course materials and requirements, and PG evaluation activities. I. SUMMARY OF ACTIVITIES:
1. AGREE Website: Development Project www.agreetrust.org • Using a user-centred development strategy and working with a website
development group specializing in building evidence-based healthcare sites (www.minervation.com ), we redeveloped and redesigned the AGREE website.
• The Website included various resources, including an online platform to complete and store individual AGREE II appraisals of PGs, “My AGREE”.
• www.agreetrust.org is the AGREE Enterprise’s website and the home for all things AGREE.
2. *NEW* (2013) – “My AGREE PLUS” • In response to user feedback and in recognizing a service gap, we enhanced the
original “My AGREE” online platform to include the functionality for completing group (multi-rater) AGREE II appraisals of practice guidelines.
• New functions include the capacity to “Contribute” to a group appraisal and to “Coordinate” a group appraisal.
• To facilitate use of “My AGREE PLUS”, several “Help” videos are available. • Visit: www.agreetrust.org and click on top right tab, “My AGREE PLUS”
3. AGREE II Training Tools (online)
• To facilitate the application and use of AGREE II, we developed two innovative, online training tools
i. AGREE II Overview Tutorial ii. AGREE II Overview Tutorial + Practice Exercise
• Visit the Resource Centre of the AGREE website: http://www.agreetrust.org/resource-centre/agree-ii-training-tools/
4. AGREE II Language Translations • As with the original AGREE Instrument, members of the international PG
community have taken the initiative to translate the AGREE II in various languages. We extend our thanks to those members for undertaking and making available the translations.
• Completed translations: i. Basques, Dutch, French, Italian, Korean, Portuguese, Portuguese
(Brazilian), Slovakian, Spanish, Thai
7
• In progress translations: i. Arabic, Chinese (Traditional), Chinese (Mandarin), Czech, Farsi
(Persian), German, Greek, Japanese, Romanian, Russian, Turkish • If you would like to undertake a translation, please contact us by emailing
[email protected] . • Visit the Resource Centre of the AGREE website to access a translation:
http://www.agreetrust.org/resource-centre/agree-ii-translations/
5. Ongoing Program of Research • As an assessment tool, AGREE II evaluates the methodological rigour used to
develop a particular practice guideline. It does not assess the clinical validity of practice guideline recommendations.
• To address the gap, the AGREE Enterprise is undertaking a program of research to develop a knowledge resource to direct the development, reporting and evaluation of practice guideline recommendation clinical credibility.
• The knowledge resource will accompany the AGREE II. • Please visit the website for updates to ongoing research work:
http://www.agreetrust.org/agree-research-projects/ As always, we welcome your feedback and suggestions. We enjoy hearing from our users and the PG community at large, so please contact us through our website or by emailing us directly via [email protected] . II. AGREE II REFERENCES: Listed below are the core references related to the AGREE II and its training tools: AGREE II: Non-Technical Paper (Main publication: Canadian Medical Association Journal; parallel publications in Journal of Clinical Epidemiology and Preventive Medicine) Brouwers M, Kho ME, Browman GP, Cluzeau F, feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. Dec 2010, 182:E839-842; doi: 10.1503/cmaj.090449 Brouwers M, Kho ME, Browman GP, Cluzeau F, feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. J Clin Epidemol. 2010, 63(12): 1308-1311 Brouwers M, Kho ME, Browman GP, Cluzeau F, feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Preventive Medicine, 2010, 51(5): 421-424 AGREE II: Technical Papers (Parts I and II) Brouwers M, Kho ME, Browman GP, Burgers J, Cluzeau F, Feder G, Fevers B, Graham ID, Hanna SE, Makarski J, on behalf of the AGREE Next Steps Consortium. Performance, usefulness and areas for improvement: Development steps toward the AGREE II – Part 1. Can Med Assoc J. 2010, 182: 1045-52
8
Brouwers MC, Kho ME, Browman GP, Burgers J, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J, on behalf of the AGREE Next Steps Consortium. Validity assessment of items and tools to support application: Development steps towards the AGREE II – Part 2. Can Med Assoc J. 2010, 182: E472-78 AGREE A3 Project, Stream 1 (Training Tools Development & Evaluation) Brouwers MC, Makarski J, Levinson A. A randomized trial to evaluate e-learning interventions designed to improve learner’s performance, satisfaction, and self-efficacy with the AGREE II. Implement Sci. 2010; 5:29 Brouwers MC, Makarski J, Durocher L, Levinson A. E-learning interventions are comparable to user’s manual in a randomized trial of training strategies for the AGREE II. Implement Sci. 2011; 6:81
9
AGREE II:
USER’S MANUAL
10
II. USER’S MANUAL: INSTRUCTIONS FOR USING THE AGREE II
This User’s Manual has been designed specifically to guide appraisers in the use of the instrument. We suggest reading the following instructions before using the instrument.
I. Preparing to Use the AGREE II
i) Accompanying Guideline Documents Before applying the AGREE II, users should first carefully read the guideline document in full. In addition to the guideline document, users should attempt to identify all information about the guideline development process prior to the appraisal. This information may be contained in the same document as the guideline recommendations or it may be summarized in a separate technical …
Collepals.com Plagiarism Free Papers
Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.
Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS
Why Hire Collepals.com writers to do your paper?
Quality- We are experienced and have access to ample research materials.
We write plagiarism Free Content
Confidential- We never share or sell your personal information to third parties.
Support-Chat with us today! We are always waiting to answer all your questions.
