The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

How Variability in the Institutional Review Board Review Process Affects Minimal-Risk Multisite Health Services Research

Laura A. Petersen, MD, MPH; Kate Simpson, MPH; Richard SoRelle, BS; Tracy Urech, MPH; and Supicha Sookanan Chitwood, MPH
[+] Article, Author, and Disclosure Information

From the Houston Veterans Affairs Health Services Research and Development Center of Excellence, Michael E. DeBakey Veterans Affairs Medical Center, and Baylor College of Medicine, Houston, Texas.

Disclaimer: Laura A. Petersen, MD, MPH, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Acknowledgment: The authors thank Eric J. Thomas, MD, MPH, for his comments on an earlier version of the article.

Grant Support: The Veterans Affairs Health Services Research and Development Investigator-Initiated Research 04-349, National Institutes of Health R01 HL079173-01, and Houston Veterans Affairs Health Services Research and Development Center of Excellence HFP90-020 funded the design and conduct of the study; the collection, management, analysis, and interpretation of the data; and the preparation of this manuscript.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-2377.

Requests for Single Reprints: Laura A. Petersen, MD, MPH, Health Services Research and Development (152), Michael E. DeBakey Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030; e-mail, mailto:laurap@bcm.edu.

Current Author Addresses: Dr. Petersen, Ms. Simpson, Mr. SoRelle, Ms. Urech, and Ms. Chitwood: Health Services Research and Development (152), Michael E. DeBakey Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030.

Author Contributions: Conception and design: L.A. Petersen.

Analysis and interpretation of the data: L.A. Petersen, K. Simpson, S.S. Chitwood.

Drafting of the article: L.A. Petersen, K. Simpson, R. SoRelle, S.S. Chitwood.

Critical revision of the article for important intellectual content: L.A. Petersen, T. Urech, K. Simpson, S.S. Chitwood.

Final approval of the article: L.A. Petersen, T. Urech, K. Simpson.

Statistical expertise: L.A. Petersen, K. Simpson.

Obtaining of funding: L.A. Petersen.

Administrative, technical, or logistic support: L.A. Petersen, T. Urech, K. Simpson, S.S. Chitwood.

Collection and assembly of data: L.A. Petersen, T. Urech, K. Simpson, R. SoRelle.

Ann Intern Med. 2012;156(10):728-735. doi:10.7326/0003-4819-156-10-201205150-00011
Text Size: A A A

Background: The Department of Health and Human Services recently called for public comment on human subjects research protections.

Objective: To assess variability in reviews across institutional review boards (IRBs) for a multisite, minimal-risk trial of financial incentives for evidence-based hypertension care and to quantify the effect of review determinations on site participation, budget, and timeline.

Design: A natural experiment occurring from multiple IRBs reviewing the same protocol for a multicenter trial (May 2005 to October 2007).

Participants: 25 Veterans Affairs (VA) medical centers.

Measurements: Number of submissions, time to approval, and costs were evaluated; patient complexity, academic affiliation, size, and location (urban or rural) between participating and nonparticipating VA medical centers were compared.

Results: Of 25 eligible VA medical centers, 6 did not meet requirements for IRB review and 2 declined to participate. Of 17 applications, 14 were approved. The process required 115 submissions, lasted 27 months, and cost close to $170 000 in staff salaries. One IRB's concern about incentivizing a particular medication recommended by national guidelines prompted a change in our design to broaden our inclusion criteria beyond uncomplicated hypertension. The change required amending the protocol at 14 sites to preserve internal validity. The IRBs that approved the protocol classified it as minimal risk. The 12 sites that ultimately participated in the trial were more likely to be urban and academically affiliated and to care for more complex patients, which limits the external validity of the trial's findings.

Limitation: Because data came from a single multisite trial in the VA system that uses a 2-stage review process, generalizability is limited.

Conclusion: Complying with IRB requirements for a minimal-risk study required substantial resources and threatened the study's internal and external validity. The current review of regulatory requirements may address some of these problems.

Primary Funding Source: Veterans Affairs Health Services Research and Development and the National Heart, Lung, and Blood Institute.


Grahic Jump Location
Figure 1.

Site selection.

IRB = institutional review board; PCP = primary care physician; PI = principal investigator; VA = Veterans Affairs.

Grahic Jump Location
Grahic Jump Location
Figure 2.

Number of days from initial submission to approval by date of initial submission.

For the 13 sites that approved the study before the modification was submitted. Sites 3, 4, and 13 did not approve the study, and the modification was incorporated into the application at site 16 before its institutional review board made an initial review decision.

* Applications were submitted to sites 10 and 11 on the same date (14 April 2006). Site 10 approved the application in 230 days, and site 11 approved it in 111 days.

Grahic Jump Location
Grahic Jump Location
Figure 3.

Effect of the IRB and R&D approval processes on study timeline.

IRB = institutional review board; R&D = Veterans Affairs Research and Development.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Overly Burdensome Research Oversight for Minimal-Risk Studies
Posted on May 22, 2012
Adam J., Rose, Investigator
Bedford VA Medical Center
Conflict of Interest: None Declared

To the Editor,

We applaud the timely and well-written study by Petersen, et al., which points out the excessively burdensome regime of research oversight as currently applied to minimal-risk studies (1). Lest readers think that the report by Petersen et al. is unusual, we wish to briefly summarize our own experience with another multi-site, minimal-risk study in the VA during 2009-2010. Our study involved 2-day site visits to three high- and three low-outlier performers on anticoagulation control, to learn more about site-level differences in organization and management that might account for differences in performance (2). Our study methods involved interviewing staff and observing clinical practice, followed by grounded thematic analysis of content. Patient data were not collected; our only human subjects were VA employees. Our efforts to identify study sites and secure approvals were fraught with difficulty. Simply locating study sites willing to participate, and securing the cooperation of a local PI at each site, often represented a formidable challenge. Similar to Petersen et al., we were unable to visit some of the worst-performing sites in the VA system, as only 3 of the 10 low-outlier sites had an IRB of record. As a consequence, instead of visiting sites with extremely poor performance, we were generally relegated to visiting sites with moderately poor performance, likely reducing our ability to contrast high- and low-outlier sites. In contrast, of 10 high-outlier sites, none lacked an IRB of record, making it much easier to find sites to visit. In the course of our one-year, $100,000 study grant, we spent over half of our funds on the process of securing approvals, which is also reminiscent of the findings of Petersen, et al. The time from IRB submission to final approval at our six study sites varied from 1-11 months. However, the total time from our first IRB submission to the completion of our final site visit spanned 18 months, in part because we were unexpectedly blocked from visiting one site at the last minute by the local employees' union, and needed to begin the IRB application process anew at another study site. This delay eroded the timeliness of our link between performance measurement (which occurred during 2007-8) and site visits (which mainly occurred during 2010), to the detriment of our study. The utility of all this extra work, in terms of protecting human research subjects, was questionable at best.

Adam Rose, MD MSc FACP Investigator, Center for Health Quality, Outcomes, and Economic Research, Bedford VA Medical Center, Bedford, MA


(1) Petersen LA, Simpson K, SoRelle R, Urech T, Chitwood SS. How variability in the Institutional Review Board process affects minimal-risk health services research. Ann Intern Med. 2012;156:728-735.

(2) Rose AJ, Petrakis BA, Callahan P, Mambourg S, Patel D, Hylek EM, Bokhour BG. Organizational Characteristics of High- and Low-Performing Anticoagulation Clinics in the Veterans Health Administration. Health Serv Res. doi: 10.1111/j.1475-6773.2011.01377.x. [Epub ahead of print]

Conflict of Interest:

None declared

Disclaimer: Dr. Rose is supported by a career development award from the VA Health Services Research and Development Service. This letter represents the opinions of the author and does not necessarily represent the official views or policies of the US Department of Veterans Affairs.

Expanding the information on IRB review
Posted on May 24, 2012
Jeffrey H., Silverstein, Associate Dean for Research
Mount Sinai School of Medicine
Conflict of Interest: None Declared

As part of a number of groups attempting to assess IRB function, I have a few questions that I believe would expand the utility and value of this fine report.

One of the key variables noted in our evaluations is the time that it takes an investigator to respond to the various IRB queries. We have attempted to quantify this by assessing the time the project is in the hands of the IRB versus the time it is in the hands of the investigators. It would be extremely useful for Dr. Petersen to provide this analysis. The format of the presentation suggests that the research team performed immediate turnaround while all of the delay was on the part of the IRB, but this is unlikely. The approaches to rectifying those issues may be different than those affecting IRB turnaround.

The investigators note that they first hired a certified IRB professional 8 months after beginning the submission process. Did that action improve the response times to IRB submission? If so, by how much and what are the implications? Would the investigators recommend that other health services researchers use the same approach?

The investigators suffered 32% attrition of potential study centers before they submitted anything to the respective IRBs. Two directors refused to participate, one center could not find an investigator and two additional investigators could not complete credentialing. Could the investigators comment on the basis for such a large failure rate and its implications for other health services researchers?

In the recent a study of IRBs, which will be presented at the CTSA- CRM Workshop at Yale (June 2,3) the quality of the protocol was a major driver of IRB throughput. For the study described by Dr. Petersen, a major problem with the review process arose out of the Office of General Counsel at one site and was eventually brought to the central VA legal counsel. The investigators indicate that they made the changes because they were concerned that these issues might arise at other sites or arise in subsequent review. It would be interesting for other health services researchers to understand whether the investigators believe that the General Counsels were incorrect in their decision such that the investigators conceded reluctantly to make these changes or whether the investigators eventually came to believe that the concerns raised improved the original study design?

Finally, the investigators noted that multiple centers expressed concerns about patient awareness of the study. It is not clear if the investigators believe this is a concept that other health services researchers should consider or that they believe that the sites that expressed this belief were misguided.

Once again, I appreciate the effort on the part of the investigators to share their experience and look forward with anticipation to additional information regarding this important topic.

Conflict of Interest:

None declared

Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.