0

The full content of Annals is available to subscribers

Subscribe/Learn More  >
Improving Patient Care |

The Effect of a Quality Improvement Collaborative to Improve Antimicrobial Prophylaxis in Surgical Patients: A Randomized Trial

Stephen B. Kritchevsky, PhD; Barbara I. Braun, PhD; Andrew J. Bush, PhD; Michele R. Bozikis, MPH; Linda Kusek, RN, MPH; John P. Burke, MD; Edward S. Wong, MD; John Jernigan, MD; Cralen C. Davis, MS; Bryan Simmons, MD, TRAPE Study Group
[+] Article and Author Information

ClinicalTrials.gov registration number: NCT00114036.

For members of the TRAPE Study Group, see the Appendix.


From Wake Forest University School of Medicine and J. Paul Sticht Center on Aging, Winston-Salem, North Carolina; The Joint Commission, Oakbrook Terrace, Illinois; University of Tennessee, Memphis, Tennessee; LDS Hospital, Salt Lake City, Utah; McGuire Veterans Affairs Medical Center and Medical College of Virginia, Richmond, Virginia; Centers for Disease Control and Prevention, Atlanta, Georgia; and Methodist Health Systems, Memphis, Tennessee.


Acknowledgment: The authors thank the study participants for their hard work and dedication to accurate data collection and Cheryl Richards, BS, RHIA; James Steinberg, MD; Mark Beezhold; Nancy Spector, RN, MSC; Richard Koss, MA; and Jerod M. Loeb, PhD.

Grant Support: By grant R01 HS11331-01A1 from the Agency for Healthcare Research and Quality and initial support from the Centers for Disease Control and Prevention.

Potential Financial Conflicts of Interest: None disclosed.

Reproducible Research Statement:Study protocol and statistical code: Available from Dr. Braun (bbraun@jointcommission.org). Data set: Available contigent on purpose through written agreement by contacting Dr. Kritchevsky (skritche@wfubmc.edu).

Requests for Single Reprints: Barbara I. Braun, PhD, Division of Quality Measurement and Research, The Joint Commission, One Renaissance Boulevard, Oakbrook Terrace, IL 60181; e-mail, bbraun@jointcommission.org.

Current Author Addresses: Dr. Kritchevsky and Mr. Davis: Wake Forest University Health Sciences, Medical Center Boulevard, Winston-Salem, NC 27157.

Dr. Braun, Ms. Bozikis, and Ms. Kusek: Division of Quality Measurement and Research, The Joint Commission, One Renaissance Boulevard, Oakbrook Terrace, IL 60181.

Dr. Bush: Department of Preventive Medicine, University of Tennessee, 66 North Pauline, Suite 633, Memphis, TN 38105.

Dr. Burke: Division of Infectious Diseases, LDS Hospital, 8th Avenue and C Street, Salt Lake City, UT 84143.

Dr. Wong: McGuire Veterans Affairs Medical Center, 1201 Broad Rock Boulevard, Richmond, VA 23249.

Dr. Jernigan: Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention, 1600 Clifton Road NE, Mailstop E-68, Atlanta, GA 30333.

Dr. Simmons: Quality Management, Methodist Health System, 1211 Union, Suite 811, Memphis, TN 38104.

Author Contributions: Conception and design: S.B. Kritchevsky, B.I. Braun, E.S. Wong, B. Simmons.

Analysis and interpretation of the data: S.B. Kritchevsky, B.I. Braun, A.J. Bush, E.S. Wong, C.C. Davis, B. Simmons.

Drafting of the article: S.B. Kritchevsky, B.I. Braun, M.R. Bozikis, E.S. Wong, J. Jernigan, C.C. Davis, B. Simmons.

Critical revision of the article for important intellectual content: S.B. Kritchevsky, J.P. Burke, E.S. Wong, J. Jernigan, B. Simmons.

Final approval of the article: S.B. Kritchevsky, J.P. Burke, E.S. Wong, J. Jernigan, C.C. Davis, B. Simmons.

Provision of study materials or patients: S.B. Kritchevsky, L. Kusek, B. Simmons.

Statistical expertise: S.B. Kritchevsky, C.C. Davis, A.J. Bush.

Obtaining of funding: S.B. Kritchevsky, B. Simmons.

Administrative, technical, or logistic support: B.I. Braun, L. Kusek, J. Jernigan.

Collection and assembly of data: B.I. Braun, L. Kusek.


Ann Intern Med. 2008;149(7):472-480. doi:10.7326/0003-4819-149-7-200810070-00007
Text Size: A A A

All 44 hospitals abstracted at least 100 eligible cases in both the baseline and remeasurement phases (Figure). The total number of eligible surgical cases included in the analysis was 4447 at baseline and 4463 at remeasurement; the mean number of cases per site was 101 (SD, 4.3) at baseline and 101 (SD, 1.9) at remeasurement. Hospitals in the intervention and feedback-only groups were similar at baseline in terms of organizational and patient characteristics (Table 2).

First Page Preview

View Large
First page PDF preview

Figures

Grahic Jump Location
Figure.
Participation in TRAPE (Trial to Reduce Antimicrobial Prophylaxis Errors).

* Number of randomly selected cases within hospitals (cluster size) was set by design to be approximately 100 at both the baseline and remeasurement data collection periods.

Grahic Jump Location
Grahic Jump Location
Appendix Figure.
Sample graph from the comparative feedback report in TRAPE (Trial to Reduce Antimicrobial Prophylaxis Errors).

Each hospital received a customized report, with a star indicating that hospital's performance on the particular measure. The vertical axis shows the percentage of surgical patients in hospitals who received prophylaxis within the recommended time frame.

Grahic Jump Location

Tables

References

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Do Quality Improvement Collaboratives Improve Antimicrobial Prophylaxis in Surgical Patients?
Posted on October 24, 2008
Stephen K. Liu
The Dartmouth Institute for Health Policy and Clinical Practice
Conflict of Interest: None Declared

TO THE Editor: The conclusions of the article by Kritchevsky et al (1) stated in the Abstract do not accurately represent the study's findings. In fact, there is substantial reason to conclude that the collaborative was of meaningful clinical value. For example, improvement was statistically significant in 4 of the 6 mean performance measures in the intervention group but in only 3 of 6 improved in the feedback-only (control) group. Moreover, improvement in the mean "all or none" measure was 15.6% greater in the intervention than the control group, which actually exceeds the 15% difference sought in the power calculation for the primary outcome. Although that particular observed difference was not statistically significant, the upper bound of the confidence interval for the all-or-none measure does not exclude a between-group advantage for the intervention group as great as 49%.

Although the article did not include the changes observed in each hospital, the wide confidence intervals for many of the pre-post measures (for example, 15.6 percentage points for the primary outcome in the control group, which is 20.8% of the baseline rate) indicate substantial heterogeneity of response to both interventions among individual hospitals, including quite large responses in some hospitals. Heterogeneity of response to either biological or social interventions is not simply "noise" (2); rather it serves as the starting point for understanding the mechanisms involved in those responses. The real value of this study therefore appears to lie in the discovery of that heterogeneity and the opportunity it offers for explaining it "“ why feedback alone or feedback-plus-collaborative was effective, for whom, and in what circumstances (3) "“ as much as in the demonstration of whether, in the aggregate, each intervention "worked." Numbers are not explanations (4), so it is fortunate that the authors have pinpointed in the discussion a number of local and external factors that could explain the differential impact of the interventions within each study group, for example, diversity of staff training and experience, variation in total and type of staff participation, and variable involvement of institutional leadership.

Unfortunately, however, the study was not designed to explain how the best individual hospital performers differed from the worst and why. For example, we have no way of knowing whether the history of successful improvement efforts differed among participating hospitals; or exactly which staff participated in the collaborative and why, and how were they involved in the care processes being changed.

Changing healthcare work is a complex, context-bound social process. Evaluation of any improvement intervention therefore requires clarity regarding the theory that underlies its selection; recognition of the contributions of individual stakeholders to its effects; awareness of the sequence and execution of its individual steps; recognition of the power relationships among people involved in its implementation; understanding of the multiple realities that affect its execution: timing, culture, resource allocation, staffing, competing priorities; awareness that most interventions start by being adapted to fit local circumstances, and that the interventions and the contexts in which they are implemented evolve over time in response to the resulting changes. For their findings to be meaningful, future studies of improvement interventions will need to include these elements in their study designs (5).

References

1) Kritchevsky SB, Braun BI, Bush AJ, Bozikis MR, Kusek L, Burke JP, et al. The effect of a quality improvement collaborative to improve antimicrobial prophylaxis in surgical patients. A randomized trial. Ann Intern Med 2008;149:472-08.

2) Heng HHQ. The conflict between complex systems and reductionism. JAMA 2008;300:1580-1.

3) Pawson R, Greenhalgh T, Haravey G, Walshe K. Realist review "“ a new method of systematic review designed for complex policy interventions. J Health Services Res & Policy. 2005;10;21-34.

4) Vandenbroucke JP. Observational research, randomised trials, and two views of medical science. PLoS Medicine 2008;5:e67.

5) Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S. Publication guidelines for improvement studies in health care: evaluation of the SQUIRE project. Ann Intern Med (in press).

Conflict of Interest:

None declared

suggestions to the authors
Posted on November 23, 2008
Daisson José Trevisol
Universidade Federal do Rio Grande do Sul
Conflict of Interest: None Declared

Response to "The Effect of a Quality Improvement Collaborative to Improve Antimicrobial Prophylaxis in Surgical Patients"

Antimicrobial prophylaxis is important to control the occurrence of postoperative infection caused by surgical site, serving therefore to delay or eliminate microbial growth. For that purpose, care in the selection, timing and duration of antibiotic is necessary as a way to reduce morbidity associated with invasive procedures.

Kritchevsky SB et al. evaluated the effects of a quality improvement collaborative on preoperative antimicrobial prophylaxis in hospitals that received support from a group of specialists versus hospitals that did not receive such assistance.

However, we noticed some methodological issues that might have affected the results. The volunteer bias, usually detected among human being, was also present among the clustered hospitals enrolled in the study. In this case, the entire studied population was more willing to change and to improve the quality of antibiotic prophylaxis. The baseline data confirmed that the performance of the intervention and control groups was already high for most of the variables related to the preoperative antimicrobial prophylaxis. So, there was not enough room for improvement in the re-measurement phase. Moreover, the feedback report was a stimulus sufficient to promote change in timing, duration, but there was no meaningful increase in the use of a single preoperative dose of antimicrobials.

Why the investigators did not use a more robust outcome, such as the rate of postoperative infection? It would be easy to standardize a protocol and measure a direct outcome. The final target should be the reduction in the infection rate by minimizing the mistakes in the preoperative antimicrobial prophylaxis.

Conflict of Interest:

None declared

Response to Dr. Liu and colleagues
Posted on December 2, 2008
Stephen B Kritchevsky
Wake Forest University School of Medicine and J. Paul Sticht Center on Aging
Conflict of Interest: None Declared

We thank Dr. Liu and colleagues for their comments regarding our recent article. They made several statements about our results that readers less familiar with the article may find misleading. While both groups showed improvements in many aspects of antimicrobial prophylaxis, there were no significant differences in the extent of improvement between the two groups on any indicator. Our conclusion is correct in this regard. Our study was designed to examine the absolute difference in change between the two groups not the relative difference in change, thus the proper statistic to evaluate the effect of participation in the collaborative group for the "all-or-none" indicator shows a 6.3% greater improvement in the active intervention (95% CI, -7.3%,19.8%).

We agree that improvement interventions are complex social processes that should be theory-based and explained in context using evaluation models drawn from social science (1, 2). However, we disagree with the statement that the real value of this study appears to lie in the discovery of the heterogeneity and the opportunity it offers for explaining it. Participants in all randomized studies, be they patients or organizations, exhibit heterogeneity. One of the greatest values of a randomized trial design is the ability to answer the question of effectiveness in the face of heterogeneity, thereby addressing the expectation of improvement for the next patient (or organization) that adopts the evaluated treatment. We agree that case studies of success can be extremely helpful in stimulating quality improvement, but the fact that we do not know which mechanisms worked within which circumstances does not negate the value of the cluster randomized trial as an evaluative methodology.

There is much to be done to improve the quality of health care delivery and quality improvement collaboratives have a prominent role. In a recent systematic review of the impact of quality improvement collaboratives, Schouten et al. concluded that the evidence of their effectiveness is encouraging but limited; they called for studies with a balance of both process-oriented reports and rigorously controlled designs to understand why some collaboratives succeed while others have little effect on practice (3). Nevertheless, quality improvement collaboratives can be expensive to implement and tightening resources require the selection of cost-effective strategies. Data from rigorous evaluations such as that collected for the TRAPE study are needed to help organizations select among potential improvement strategies.

References

Grol R, Grimshaw J From best evidence to best practice. Lancet. 2003;11;362(9391):1225-30.

Pawson R, Tilley N. Realistic Evaluation. London, England: Sage Publications Ltd: 1997.

Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol R. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008;336:1491-1494

Conflict of Interest:

None declared

Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Buy Now

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Related Point of Care
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)