The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

Reporting Discrepancies Between the ClinicalTrials.gov Results Database and Peer-Reviewed PublicationsDiscrepancies Between ClinicalTrials.gov and Peer-Reviewed Publications

Daniel M. Hartung, PharmD, MPH; Deborah A. Zarin, MD; Jeanne-Marie Guise, MD, MPH; Marian McDonagh, PharmD; Robin Paynter, MLS; and Mark Helfand, MD, MS, MPH
[+] Article, Author, and Disclosure Information

From Oregon Health & Science University and Portland Veterans Affairs Medical Center, Portland, Oregon, and National Library of Medicine, Bethesda, Maryland.

Disclaimer: The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the National Library of Medicine or the Agency for Healthcare Research and Quality.

Acknowledgment: The authors thank Luke Middleton, BS, for developing the abstraction tool and database and Kaylee Williamson, BS, PharmD candidate, for assisting in sample validation.

Financial Support: By contract HHSN 276-2009-00128P from the National Library of Medicine and grant K12HS019456 from the Agency for Healthcare Research and Quality. Dr. Hartung received support from an Agency for Healthcare Research and Quality career development award (K12 HS019456). Dr. Zarin was funded by the intramural research program of the National Library of Medicine.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M13-0480.

Requests for Single Reprints: Daniel M. Hartung, PharmD, MPH, Oregon State University/Oregon Health & Science University, College of Pharmacy, 3303 SW Bond Avenue, CH12C, Portland, OR 97239; e-mail, hartungd@ohsu.edu.

Current Author Addresses: Dr. Hartung: Oregon State University/Oregon Health & Science University, College of Pharmacy, 3303 SW Bond Avenue, CH12C, Portland, OR 97239.

Dr. Zarin: National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894.

Dr. Guise: Oregon Health & Science University, Department of Obstetrics and Gynecology, 3181 SW Sam Jackson Park Road, Portland, OR 97239.

Dr. McDonagh and Ms. Paynter: Oregon Health & Science University, Department of Medical Informatics and Clinical Epidemiology, Mail Stop BICC, 3181 SW Sam Jackson Park Road, Portland, OR 97239.

Dr. Helfand: Portland VA Medical Center, Hospital and Specialty Care Division, Mail Code P3MED, Portland, OR 97239.

Author Contributions: Conception and design: D.M. Hartung, D.A. Zarin, M. McDonagh, M. Helfand.

Analysis and interpretation of the data: D.M. Hartung, D.A. Zarin, J.M. Guise, M. McDonagh, R. Paynter, M. Helfand.

Drafting of the article: D.M. Hartung, J.M. Guise, M. McDonagh, M. Helfand.

Critical revision of the article for important intellectual content: D.M. Hartung, D.A. Zarin, J.M. Guise, M. McDonagh, M. Helfand.

Final approval of the article: D.M. Hartung, D.A. Zarin, J.M. Guise, M. Helfand.

Provision of study materials or patients: D.M. Hartung.

Statistical expertise: D.M. Hartung, M. Helfand.

Obtaining of funding: M. Helfand.

Administrative, technical, or logistic support: D.A. Zarin, J.M. Guise, M. Helfand.

Collection and assembly of data: D.M. Hartung, R. Paynter.

Ann Intern Med. 2014;160(7):477-483. doi:10.7326/M13-0480
Text Size: A A A

Background: ClinicalTrials.gov requires reporting of result summaries for many drug and device trials.

Purpose: To evaluate the consistency of reporting of trials that are registered in the ClinicalTrials.gov results database and published in the literature.

Data Sources: ClinicalTrials.gov results database and matched publications identified through ClinicalTrials.gov and a manual search of 2 electronic databases.

Study Selection: 10% random sample of phase 3 or 4 trials with results in the ClinicalTrials.gov results database, completed before 1 January 2009, with 2 or more groups.

Data Extraction: One reviewer extracted data about trial design and results from the results database and matching publications. A subsample was independently verified.

Data Synthesis: Of 110 trials with results, most were industry-sponsored, parallel-design drug studies. The most common inconsistency was the number of secondary outcome measures reported (80%). Sixteen trials (15%) reported the primary outcome description inconsistently, and 22 (20%) reported the primary outcome value inconsistently. Thirty-eight trials inconsistently reported the number of individuals with a serious adverse event (SAE); of these, 33 (87%) reported more SAEs in ClinicalTrials.gov. Among the 84 trials that reported SAEs in ClinicalTrials.gov, 11 publications did not mention SAEs, 5 reported them as zero or not occurring, and 21 reported a different number of SAEs. Among 29 trials that reported deaths in ClinicalTrials.gov, 28% differed from the matched publication.

Limitation: Small sample that included earliest results posted to the database.

Conclusion: Reporting discrepancies between the ClinicalTrials.gov results database and matching publications are common. Which source contains the more accurate account of results is unclear, although ClinicalTrials.gov may provide a more comprehensive description of adverse events than the publication.

Primary Funding Source: Agency for Healthcare Research and Quality.


Grahic Jump Location

Summary of evidence search and selection.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Reproducible Research on Discrepant Reporting of Results
Posted on April 8, 2014
Jessica E. Becker, AB; Joseph S. Ross, MD, MHS
Yale University School of Medicine, New Haven, CT
Conflict of Interest: Funding/support and role of the sponsor: This letter to the editor was not supported by any external grants or funds. Dr. Ross receives research support from Medtronic, Inc. and Johnson & Johnson, Inc. to develop methods of clinical trial data sharing, from the Centers for Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting, from the Pew Charitable Trusts to study FDA pharmaceutical regulation, and from the Food and Drug Administration (FDA) to develop methods for post-market surveillance of medical devices. Dr. Ross is supported by the National Institute on Aging (K08 AG032886) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program.
Conflicts of interest: Dr. Ross reports that he is a member of a scientific advisory board for FAIR Health, Inc.
To The Editor:

Hartung and colleagues examined a random sample of phase 3 and 4 clinical trials reporting results on ClinicalTrials.gov and compared reported results with corresponding journal publications, finding high rates of inconsistency in the reporting of outcome descriptions, outcome values, and adverse events (1). We performed a similar study published recently in JAMA that had strikingly similar results (2), despite examining a distinct sample of trial registrations and publications (note: we explicitly focused on trials registered on ClinicalTrials.gov and published in high-impact journals).

In their study, Hartung found 15% of trials reported discrepant primary outcome descriptions between ClinicalTrials.gov and corresponding publications, while 20% inconsistently reported primary outcome results values. Similarly, we found that 15% of primary outcomes were not described consistently between ClinicalTrials.gov and corresponding publications, and 16% of comparable primary results values were discordant between the two sources.

Hartung also found that 35% of trials had reporting discrepancies for serious adverse events between ClinicalTrials.gov and publications. Not reported in our article, we too found that 39% of trials had discrepant serious adverse event reporting between sources. They elaborated that the inconsistencies were frequently due to underreporting or omission of adverse events from publications, a finding that held true in our sample as well. Whereas Hartung found 37% of trials with at least one other adverse event in ClinicalTrials.gov to have discrepant adverse event reporting between sources, we found slightly higher rates of these discrepancies (48% of all trials).

Results reporting on ClinicalTrials.gov began only after the 2007 Food and Drug Administration Amendments Act, and these studies offer important insights into ways that the system can be iteratively improved. Physicians, patients and investigators cannot know whether results reported on ClinicalTrials.gov or the results reported in the published article are accurate, and there are both reasonable and non-reasonable explanations for the discrepant results between the two sources. However, for results reporting to be effective, these discrepancies need to be fixed, and prevented in the future. At the very least, while the trial results reported on ClinicalTrials.gov and in corresponding publications are not yet consistent, at least research studies examining this issue are.

1. Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M. Reporting Discrepancies Between the ClinicalTrials.gov Results Database and Peer-Reviewed Publications. Ann Intern Med. 2014;160(7):477-83.
2. Becker JE, Krumholz HM, Ben-Josef G, Ross JS. Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA. 2014;311(10):1063-5.
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Journal Club
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.