Laura A. Petersen, MD, MPH; Kate Simpson, MPH; Richard SoRelle, BS; Tracy Urech, MPH; Supicha Sookanan Chitwood, MPH
Disclaimer: Laura A. Petersen, MD, MPH, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Acknowledgment: The authors thank Eric J. Thomas, MD, MPH, for his comments on an earlier version of the article.
Grant Support: The Veterans Affairs Health Services Research and Development Investigator-Initiated Research 04-349, National Institutes of Health R01 HL079173-01, and Houston Veterans Affairs Health Services Research and Development Center of Excellence HFP90-020 funded the design and conduct of the study; the collection, management, analysis, and interpretation of the data; and the preparation of this manuscript.
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-2377.
Requests for Single Reprints: Laura A. Petersen, MD, MPH, Health Services Research and Development (152), Michael E. DeBakey Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030; e-mail, mailto:email@example.com.
Current Author Addresses: Dr. Petersen, Ms. Simpson, Mr. SoRelle, Ms. Urech, and Ms. Chitwood: Health Services Research and Development (152), Michael E. DeBakey Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030.
Author Contributions: Conception and design: L.A. Petersen.
Analysis and interpretation of the data: L.A. Petersen, K. Simpson, S.S. Chitwood.
Drafting of the article: L.A. Petersen, K. Simpson, R. SoRelle, S.S. Chitwood.
Critical revision of the article for important intellectual content: L.A. Petersen, T. Urech, K. Simpson, S.S. Chitwood.
Final approval of the article: L.A. Petersen, T. Urech, K. Simpson.
Statistical expertise: L.A. Petersen, K. Simpson.
Obtaining of funding: L.A. Petersen.
Administrative, technical, or logistic support: L.A. Petersen, T. Urech, K. Simpson, S.S. Chitwood.
Collection and assembly of data: L.A. Petersen, T. Urech, K. Simpson, R. SoRelle.
Petersen L., Simpson K., SoRelle R., Urech T., Chitwood S.; How Variability in the Institutional Review Board Review Process Affects Minimal-Risk Multisite Health Services Research. Ann Intern Med. 2012;156:728-735. doi: 10.7326/0003-4819-156-10-201205150-00011
Download citation file:
Published: Ann Intern Med. 2012;156(10):728-735.
The Department of Health and Human Services recently called for public comment on human subjects research protections.
To assess variability in reviews across institutional review boards (IRBs) for a multisite, minimal-risk trial of financial incentives for evidence-based hypertension care and to quantify the effect of review determinations on site participation, budget, and timeline.
A natural experiment occurring from multiple IRBs reviewing the same protocol for a multicenter trial (May 2005 to October 2007).
25 Veterans Affairs (VA) medical centers.
Number of submissions, time to approval, and costs were evaluated; patient complexity, academic affiliation, size, and location (urban or rural) between participating and nonparticipating VA medical centers were compared.
Of 25 eligible VA medical centers, 6 did not meet requirements for IRB review and 2 declined to participate. Of 17 applications, 14 were approved. The process required 115 submissions, lasted 27 months, and cost close to $170 000 in staff salaries. One IRB's concern about incentivizing a particular medication recommended by national guidelines prompted a change in our design to broaden our inclusion criteria beyond uncomplicated hypertension. The change required amending the protocol at 14 sites to preserve internal validity. The IRBs that approved the protocol classified it as minimal risk. The 12 sites that ultimately participated in the trial were more likely to be urban and academically affiliated and to care for more complex patients, which limits the external validity of the trial's findings.
Because data came from a single multisite trial in the VA system that uses a 2-stage review process, generalizability is limited.
Complying with IRB requirements for a minimal-risk study required substantial resources and threatened the study's internal and external validity. The current review of regulatory requirements may address some of these problems.
Veterans Affairs Health Services Research and Development and the National Heart, Lung, and Blood Institute.
Adam J., Rose, Investigator
Bedford VA Medical Center
May 22, 2012
Overly Burdensome Research Oversight for Minimal-Risk Studies
To the Editor,
We applaud the timely and well-written study by Petersen, et al., which points out the excessively burdensome regime of research oversight as currently applied to minimal-risk studies (1). Lest readers think that the report by Petersen et al. is unusual, we wish to briefly summarize our own experience with another multi-site, minimal-risk study in the VA during 2009-2010. Our study involved 2-day site visits to three high- and three low-outlier performers on anticoagulation control, to learn more about site-level differences in organization and management that might account for differences in performance (2). Our study methods involved interviewing staff and observing clinical practice, followed by grounded thematic analysis of content. Patient data were not collected; our only human subjects were VA employees. Our efforts to identify study sites and secure approvals were fraught with difficulty. Simply locating study sites willing to participate, and securing the cooperation of a local PI at each site, often represented a formidable challenge. Similar to Petersen et al., we were unable to visit some of the worst-performing sites in the VA system, as only 3 of the 10 low-outlier sites had an IRB of record. As a consequence, instead of visiting sites with extremely poor performance, we were generally relegated to visiting sites with moderately poor performance, likely reducing our ability to contrast high- and low-outlier sites. In contrast, of 10 high-outlier sites, none lacked an IRB of record, making it much easier to find sites to visit. In the course of our one-year, $100,000 study grant, we spent over half of our funds on the process of securing approvals, which is also reminiscent of the findings of Petersen, et al. The time from IRB submission to final approval at our six study sites varied from 1-11 months. However, the total time from our first IRB submission to the completion of our final site visit spanned 18 months, in part because we were unexpectedly blocked from visiting one site at the last minute by the local employees' union, and needed to begin the IRB application process anew at another study site. This delay eroded the timeliness of our link between performance measurement (which occurred during 2007-8) and site visits (which mainly occurred during 2010), to the detriment of our study. The utility of all this extra work, in terms of protecting human research subjects, was questionable at best.
Adam Rose, MD MSc FACP Investigator, Center for Health Quality, Outcomes, and Economic Research, Bedford VA Medical Center, Bedford, MA
(1) Petersen LA, Simpson K, SoRelle R, Urech T, Chitwood SS. How variability in the Institutional Review Board process affects minimal-risk health services research. Ann Intern Med. 2012;156:728-735.
(2) Rose AJ, Petrakis BA, Callahan P, Mambourg S, Patel D, Hylek EM, Bokhour BG. Organizational Characteristics of High- and Low-Performing Anticoagulation Clinics in the Veterans Health Administration. Health Serv Res. doi: 10.1111/j.1475-6773.2011.01377.x. [Epub ahead of print]
Disclaimer: Dr. Rose is supported by a career development award from the VA Health Services Research and Development Service. This letter represents the opinions of the author and does not necessarily represent the official views or policies of the US Department of Veterans Affairs.
Jeffrey H., Silverstein, Associate Dean for Research
Mount Sinai School of Medicine
May 24, 2012
Expanding the information on IRB review
As part of a number of groups attempting to assess IRB function, I have a few questions that I believe would expand the utility and value of this fine report.
One of the key variables noted in our evaluations is the time that it takes an investigator to respond to the various IRB queries. We have attempted to quantify this by assessing the time the project is in the hands of the IRB versus the time it is in the hands of the investigators. It would be extremely useful for Dr. Petersen to provide this analysis. The format of the presentation suggests that the research team performed immediate turnaround while all of the delay was on the part of the IRB, but this is unlikely. The approaches to rectifying those issues may be different than those affecting IRB turnaround.
The investigators note that they first hired a certified IRB professional 8 months after beginning the submission process. Did that action improve the response times to IRB submission? If so, by how much and what are the implications? Would the investigators recommend that other health services researchers use the same approach?
The investigators suffered 32% attrition of potential study centers before they submitted anything to the respective IRBs. Two directors refused to participate, one center could not find an investigator and two additional investigators could not complete credentialing. Could the investigators comment on the basis for such a large failure rate and its implications for other health services researchers?
In the recent a study of IRBs, which will be presented at the CTSA- CRM Workshop at Yale (June 2,3) the quality of the protocol was a major driver of IRB throughput. For the study described by Dr. Petersen, a major problem with the review process arose out of the Office of General Counsel at one site and was eventually brought to the central VA legal counsel. The investigators indicate that they made the changes because they were concerned that these issues might arise at other sites or arise in subsequent review. It would be interesting for other health services researchers to understand whether the investigators believe that the General Counsels were incorrect in their decision such that the investigators conceded reluctantly to make these changes or whether the investigators eventually came to believe that the concerns raised improved the original study design?
Finally, the investigators noted that multiple centers expressed concerns about patient awareness of the study. It is not clear if the investigators believe this is a concept that other health services researchers should consider or that they believe that the sites that expressed this belief were misguided.
Once again, I appreciate the effort on the part of the investigators to share their experience and look forward with anticipation to additional information regarding this important topic.
to gain full access to the content and tools.
Learn more about subscription options.
Register Now for a free account.
Research and Reporting Methods.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only