JAOA • Vol 107 • No 8 • August 2007 • 321-324
Evidence-Based Medicine, Part 6. An Introduction to Critical Appraisal of Clinical Practice Guidelines
Brent W. Sanderlin, DO;
Nashila AbdulRahim, OMS IV
From the Department of Family Medicine at the University of North Texas
Health Science Center—Texas College of Osteopathic Medicine in Fort
Worth.
Address correspondence to Brent W. Sanderlin, DO, Department of Family
Medicine, Texas College of Osteopathic Medicine, University of North Texas
Health Science Center, 3400 Camp Bowie Blvd, Fort Worth, TX 76107-2729.E-mail:
bsanderl{at}hsc.unt.edu
This article provides an introductory step-by-step process to appraise
clinical practice guidelines. The authors introduce these principles using a
systematic approach and case-based format. The process of assessing the
validity of clinical practice guidelines, determining their importance, and
applying them to an individual patient is reviewed. The concepts of study
population homogeneity, equal treatment, and study completeness are discussed
to help physicians determine the validity of clinical practice guidelines.
Finally, information that is learned from the previously mentioned steps is
applied to patient care. Study generalizability and the role of patient
values, expectations, and concerns are also addressed. The skills learned from
appraising clinical practice guidelines in the manner outlined provides a
solid basis for life-long learning and improved patient care.
Physicians, who use their clinical skills to treat patients and diagnose
diseases on a daily basis, must be informed of current clinical practice
guidelines and standards of care in order to provide their patients with the
best care possible. However, because such guidelines may have different
recommendations depending on who wrote or sponsored the guideline, they may
not be applicable to any given patient. Therefore, physicians must remain
cognizant of the rationale used to develop such guidelines—in particular
the evidence base used—while at the same time assessing the guidelines'
validity and applicability. One such tool to develop clinicians' skills in
evaluating guidelines is evidence-based medicine (EBM).
In this article, we introduce a strategy for busy physicians, physician
residents, and medical students to critically assess clinical practice
guidelines. In-depth details of research methods are beyond the scope of this
introductory series on EBM. Readers are encouraged to seek further training on
these topics with supplemental learning opportunities and continuing medical
education. Finally, the clinical scenario described has been simplified to
provide readers with an illustrative example for the general concepts
introduced.
 |
Searching the Evidence
|
|---|
The initial step required for a critical review of clinical practice
guidelines involves evaluating the document's stated purpose (eg, diagnosis,
prevention, symptom control). Secondarily, physicians practicing EBM are
concerned with evaluating how closely the guideline's purpose is aligned with
a particular patient's needs (Figure
1).
Guidelines are valuable insofar as they provide clear and practical
recommendations for a clinical challenge. A guideline, to prove relevant, must
be supported by evidence that assesses the risks, benefits, and costs of the
proposed intervention. A guideline should also explain how its recommendations
can be implemented, regularly reviewed, and updated to ensure accurate and
up-to-date information is provided.
|
Validity of Clinical Practice Guidelines
|
|---|
There are a number of items to take into consideration when appraising a
clinical practice guideline.
Greenhalgh1 suggests
10 questions to use when reviewing a guideline
(Figure 2). For
example, Greenhalgh asks, "Did the preparation and publication of this
guideline involve a significant conflict of interest?" If the guideline
developers received financial support from a pharmaceutical company that
manufactures a medication recommended in the guideline, those guideline
authors may be unconsciously (or consciously) biased with their specified
recommendations. Similar to how authors of articles published in medical
journals are required to disclose financial
relationships,2,3
members of guideline development teams should divulge any special interests
they have related to the topic of the clinical practice guideline.
Although conflict-of-interest statements are often absent from
guidelines,4 a good
guideline will disclose any potential bias so physicians can better determine
the strength of the guidelines. If a physician finds that there is a
significant conflict of interest in either the preparation or publication of
the guideline, he or she should begin a new search for recommendations.
Additional questions that might be asked by a physician practicing EBM
include the following:
- How strong is the evidence used to support the clinical practice
guidelines?
The quality or "strength" of clinical practice guidelines can
vary significantly as a result of the evidence used in their development. Such
sources can range from well-designed randomized controlled trials (RCTs) to
expert opinions.5
The strongest evidence, according to the US Preventive Services Task
Force,6 comes from
well-designed experimental studies, such as RCTs. And yet, guideline authors
can undermine the value of their work by including in their evidence base RCTs
that fail to account for a small sample size, patients lost to follow-up,
possible biases, and results that cannot be replicated in other trials. Where
possible, physicians should critically review the individual studies used by
guideline authors in making their recommendations for standards of care.
Avoiding adverse events or high costs by using inexpensive, low-risk
treatments can strengthen clinical practice guideline recommendations. Such
guidelines provide alternative choices that may be effective yet inexpensive
with less risk to the patient for adverse effects. For example, the Joint
National Committee on Prevention, Detection, Evaluation, and Treatment of High
Blood Pressure7
recommends the use of diuretics, which tend to be safer and less expensive
than many alternatives.
- Are the guidelines objective?
Outside involvement of a researcher with appropriate and unbiased
analytical skills, in addition to expert validation, provides a more objective
methodology for guideline development. Development of such guidelines must
also take ethical, cultural, social, and personal values as well as societal
views into account. Clinical practice guidelines should be based on all
evidence related to the guideline's topic; basing recommendations solely on
expert opinion is subjective and possibly misleading. The appraisal processes
for specific types of studies have been reviewed in detail in other articles
associated with this
series.8-11
- Are the methods of the studies used homogenous?
Not surprisingly, the validity of studies' results determines the quality
of the guideline's recommendations. However, differences in population
characteristics, clinical settings, methods of administering interventions,
and follow-up rates can lead to somewhat undesirable heterogeneous study
results—which may then, in turn, lead to less valid guideline
recommendations.12
Clinical practice guidelines that are based on several studies with homogenous
methods and complementary results are stronger
(Figure 3).
A summary of the grades of guideline recommendations based on study design
is available in Figure
4.5
- Are the study subjects different in any significant way from your
specific patient?
The findings of a valid, important study are considered generalizable when
patients to be treated are similar to those described in the study. Similarly,
recommendations made in clinical practice guidelines cannot be generalized if
the data or evidence on which they are based is derived from a population that
significantly differs from physicians' patients. Such information can be
accessed by reviewing the population characteristics of the studies used to
create clinical practice guidelines.
- Are the guideline's recommendations an accurate reflection of the
appraised evidence?
Final recommendations of clinical practice guidelines should be an accurate
reflection of the appraised evidence. That is, the recommendations should be
weighted on the evidence that has been critically scrutinized and appraised.
More importantly, these valid recommendations must be amenable to medical
practice variants and the compromises associated with realworld practice (eg,
guidelines should be flexible in various clinical settings and relevant to
clinical practice).
View larger version (41K):
[in this window]
[in a new window]
|
Figure 4. Ratings of guideline recommendations based on the study designs of
articles used to support the guideline's findings. Abbreviations: CIs,
confidence intervals; NNT, number needed to treat; RCTs, randomized controlled
trials. Adapted from Wilson MC et al. JAMA.
1995;274:1630-1632.1
Copyright 1995, American Medical Association. All rights
reserved.
|
|
|
Practical Use
|
|---|
A guideline's generalizability is directly related to its inclusion and
exclusion criteria (eg, guidelines based on more stringent criteria are less
generalizable to a
population).5 In
addition, applicability of clinical practice guidelines must consider the
burden of disease, the patient's values and unique circumstances, and any
practical barriers to guideline implementation. If the burden of disease is
very low, any intervention's risk-to-benefit ratio becomes less advantageous
to the
patient.12
Patients and physicians must work together to ensure both parties are
making informed decisions. Insurance restrictions, availability of
interventions, and patient factors and limitations (eg, financial, logistic)
are common practical barriers to guideline implementation. All of these
factors must be addressed to ensure a guideline is applicable to any given
patient (Figure 5).
However, these factors do not affect the basic evidence or scientific validity
of a
guideline.13
|
Conclusion
|
|---|
Although most clinicians are already incorporating EBM principles in their
practices, often instinctively, some physicians may require a more organized
approach to integrating this relatively new model of self-education. Improved
comfort levels and true expertise in the practice of EBM are the result of
additional education, repetition, and self-assessment. The principles of EBM
allow physicians to stay informed while also improving the quality of the
information communicated to patients during patient encounters. The systematic
approach that is used to appraise clinical practice guidelines is but one step
in practicing EBM. Remember, the goal is always to provide the best care
possible to patients—using one's clinical expertise to address patient
values and expectations for treatment.
|
Footnotes
|
|---|
[Editor's note: This article is part 6 of a six-article series intended
to introduce the principles of evidence-based medicine (EBM) to busy
clinicians, physician residents, and medical students. Because the application
of EBM is a career-long process, further training is needed beyond the
information provided within this article and series. A foundation of knowledge
about research methods is critical in understanding EBM; however, such
details, though introduced, are beyond the scope of this series.]
Submitted March 13, 2007;
revision received June 14, 2007;
accepted June 18, 2007.
|
References
|
|---|
1. Greenhalgh T. How to Read a Paper: The Basics of
Evidence-Based Medicine. 3rd ed. Malden, Mass: Blackwell
Publishing Ltd; 2006.2. Uniform Requirements for Manuscripts Submitted to Biomedical
Journals: Writing and Editing for Biomedical Publication. February 2006.
International Committee of Medical Journal Editors Web site. Available at:
http://www.icmje.org/.
Accessed August 15, 2007.
3. Ethical and legal considerations. In: Iverson C, ed. AMA
Manual of Style: A Guide for Authors and Editors. 10th ed. New
York, NY: Oxford University Press, Inc; 2007.
4. Taylor R, Giles J. Cash interests taint drug advice.
Nature. 2005;437:1070
-1071.[Medline]
5. Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt G, for the
Evidence-Based Medicine Working Group. Users' guides to the medical
literature. VIII. B. How to use clinical guidelines. What are the
recommendations and will they help you in caring for your patients?
JAMA. 1995;274:1630
-1632.[Medline]
6. Agency for Healthcare Research and Quality. United States
Preventive Services Task Force. Available at:
http://www.ahrq.gov/clinic/uspstfix.htm.
Accessed August 12, 2007.
7. Chobanian AV, Bakris GL, Black HR. The Seventh Report of the Joint
National Committee on Prevention, Detection, Evaluation, and Treatment of High
Blood Pressure: the JNC 7 report [published correction appears in
JAMA. 2003;290:197]. JAMA.2003; 289:2560
-2573.[Abstract/Free Full Text]
8. Cardarelli R, Virgilio RF, Taylor L. Evidence-based medicine, part
2. An introduction to critical appraisal of articles on therapy. J Am
Osteopath Assoc. 2007;107:299-303. Available at:
http://www.jaoa.org/cgi/content/full/107/8/299.
9. Schranz DA, Dunn MA. Evidence-based medicine, part 3. An
introduction to critical appraisal of articles on diagnosis. J Am
Osteopath Assoc. 2007;107:304-309. Available at:
http://www.jaoa.org/cgi/content/full/107/8/304.
10. Cardarelli R, Seater MM. Evidence-based medicine, part 4. An
introduction to critical appraisal of articles on harm. J Am Osteopath
Assoc. 2007;107:310-314. Available at:
http://www.jaoa.org/cgi/content/full/107/8/310.
11. Cardarelli R, Oberdorfer JR. Evidence-based medicine, part 5. An
introduction to critical appraisal of articles on prognosis. J Am
Osteopath Assoc. 2007;107:315-319. Available at:
http://www.jaoa.org/cgi/content/full/107/8/315.
12. Guyatt G, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, for
the Evidence-Based Medicine Working Group. Users' guide to the medical
literature. IX. A method for grading health care recommendations.
JAMA. 1995;274:1800
-1804.[Medline]
13. Straus SE, Richardson WS, Glasziou P, Haynes RB.
Evidence-Based Medicine: How to Practice and Teach
EBM. 3rd ed. London: Churchill Livingstone;2005
.
Top
Searching the Evidence
Validity of Clinical Practice...
