Karen A. Robinson, PhD; Steven N. Goodman, MD, MHS, PhD
Robinson KA, Goodman SN. A Systematic Examination of the Citation of Prior Research in Reports of Randomized, Controlled Trials. Ann Intern Med. 2011;154:50-55. doi: 10.7326/0003-4819-154-1-201101040-00007
Download citation file:
Published: Ann Intern Med. 2011;154(1):50-55.
A randomized, controlled trial (RCT) should not be started or interpreted without accounting for evidence from preceding RCTs addressing the same question. Research has suggested that evidence from prior trials is often not accounted for in reports of subsequent RCTs.
To assess the extent to which reports of RCTs cite prior trials studying the same interventions.
Meta-analyses published in 2004 that combined 4 or more trials were identified; within each meta-analysis, the extent to which each trial report cited the trials that preceded it by more than 1 year was assessed.
The proportion of prior trials that were cited (prior research citation index), the proportion of the total participants from prior trials that were in the cited trials (sample size citation index), and the absolute number of trials cited were calculated.
227 meta-analyses were identified, comprising 1523 trials published from 1963 to 2004. The median prior research citation index was 0.21 (95% CI, 0.18 to 0.24), meaning that less than one quarter of relevant reports were cited. The median sample size citation index (0.24 [CI, 0.21 to 0.27]) was similar, suggesting that larger trials were not selectively cited. Of the 1101 RCTs that had 5 or more prior trials to cite, 254 (23%) cited no prior RCTs and 257 (23%) cited only 1. The median number of prior cited trials was 2, which did not change as the number of citable trials increased. The mean number of preceding trials cited by trials published after 2000 was 2.4, compared with 1.5 for those published before 2000 (P < 0.001).
The investigators could not ascertain why prior trials were not cited, and noncited trials may have been taken into account in the trial design and proposal stages.
In reports of RCTs published over 4 decades, fewer than 25% of preceding trials were cited, comprising fewer than 25% of the participants enrolled in all relevant prior trials. A median of 2 trials was cited, regardless of the number of prior trials that had been conducted. Research is needed to explore the explanations for and consequences of this phenomenon. Potential implications include ethically unjustifiable trials, wasted resources, incorrect conclusions, and unnecessary risks for trial participants.
Boxes span the interquartile range; the white line in each box is the median. Whiskers indicate the upper and lower adjacent values. The circles above the whiskers are individual outliers. The black horizontal line is the median number of trials cited.
TS=meta-analy* OR TS=metanaly*, DocTypes=review OR article, 2004
TS=random* OR TS = RCT*
#1 AND #2
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Welch Medical Library, Johns Hopkins University School of Medicine, Baltimore, MD 21205
July 29, 2011
Your Informationist is "In"
This article is beneficial to informationists, health science librarians who work within the context of research and clinical teams, in our pursuit to assist healthcare providers, scientists and scholars advance their clinical practice and research efforts . With the proliferation of medical information come two time-consuming tasks: searching for all relevant literature from numerous sources, and critically appraising studies for the purposes of basing conclusions, new ideas and support for hypotheses. Informationists facilitate this process by participating in the production of systematic reviews of the literature, which requires the rigorous examination and summation of studies that answer specific clinical questions.
The opportunities to search for information occur at different stages in the research process, from the designing of a trial protocol to the writing of proposals for research grants. Underlying requests to satisfy medical information needs are these familiar questions: "Has anyone done this before, and if so, what was done and how?" and "How can I be sure I haven't missed anything?" Information professionals excel in facilitating the formulation of questions, efficiently identifying appropriate resources and transforming user's questions into competent, effective search queries to find relevant information from electronic databases. Informationists can also provide reasonable certainty that the information sought does not exist. How information is used--or not--relies heavily on how thoroughly the first two tasks in the process are accomplished.
While the authors assert that more research is needed to explore the reasons why prior trials go uncited, informationists also see this study as bolstering the case for their continued role in multidisciplinary, collaborative clinical and research teams, and for their participation in scholarship practice endeavors. We are eager to join in these efforts, and offer our time, skills and expertise. It seems we have been here before, not long ago, when we collectively wondered if medical librarians could have helped save a life [2,3].
Blair Anton, MLIS, MS Welch Medical Library, Johns Hopkins University School of Medicine, Baltimore, MD 21205
Nancy K. Roderer, MLS Welch Medical Library, Johns Hopkins University School of Medicine, Baltimore, MD 21205
1. Robinson, KA, Goodman, SN. A Systematic Examination of the Citation of Prior Research in Reports of Randomized Controlled Trials. Ann Intern Med. 2011 Jan; 154(1): 50-54
2. Holst, Ruth. Expert Searching. J Med Libr Assoc. 2005 Jan; 93(1): 41
3. Savulescu, J., Spriggs, M. The Hexamethonium Asthma Study and the Death of a Normal Volunteer in Research. J Med Ethics. 2002 Feb; 28(1): 3- 4
Research and Reporting Methods.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only