Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that she has no financial relationships or interests to disclose. Darren B. Taichman, MD, PhD, Executive Deputy Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Deborah Cotton, MD, MPH, Deputy Editor, reports that she has no financial relationships or interest to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Catharine B. Stack, PhD, MS, Deputy Editor for Statistics, reports that she has stock holdings in Pfizer.
This article was published at www.annals.org on 22 December 2016.
The Editors. Discrepancies Between Prespecified and Reported Outcomes. Ann Intern Med. 2016;164:374-375. doi: 10.7326/L15-0615
Download citation file:
Published: Ann Intern Med. 2016;164(5):374-375.
Published at www.annals.org on 22 December 2015
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Kyle Lehenbauer, MD FACP
Veterans Affairs Medical Center, Minneapolis
December 29, 2015
Your response makes some valid points but I think one of the main aims of the COMPare group is to point out that having too many outcomes and especially switching outcomes increases the probability of having a good result based more on random chance than the actual intervention. Being able to switch outcomes or not report pre-specified outcomes makes a p value useless. How can you publish a probability that the results were due to random chance after the team has seen the results and gotten to choose which results they publish?
Editor in Chief, Annals of Internal Medicine
February 12, 2016
Conflict of Interest:
Employment: Editor in Chief, Annals of Internal Medicine
Review Process for Clinical Trials
The editors of Annals of Internal Medicine agree with Dr. Lehenbauer that having too many outcomes or switching outcomes can increase the possibility of detecting a favorable outcome in a clinical trial. That is why Annals’ review process for clinical trials requires the authors to submit copies of the pre-enrollment protocols with amendments to assure that the report accurately reflects the scientific and clinical intent outlined in the protocol. When the review process generates requests for authors to report outcomes not specified in the protocol or the authors choose themselves to present such outcomes, we ask authors to indicate these as post hoc or exploratory analyses. Unfortunately, there is no current global mandate that requires clinical trialists to publish or publicly post protocols prior to the start of patient enrollment. Thus, we often need to obtain the protocols from the authors. We do not rely on the trial registries alone.
Ben Goldacre, Henry Drysdale, Kamal R. Mahtani, on behalf of the COMPare Trials Project
Centre for Evidence-Based Medicine, University of Oxford, Oxford, UK
April 7, 2016
Annals needlessly using secret documents behind closed doors
We are extremely concerned by the above response from Annals’ Editor in Chief, Christine Laine. The COMPare project has assessed trials in five major journals and provided extensive documentation of outcome switching in all five, including Annals . Our figures are consistent with extensive previous published work on the prevalence of outcome switching [2,3]. Disappointingly, Annals’ editors have gone to great lengths to avoid open discussion of this issue , argued for the use of publicly inaccesible protocols, and argued against using registers as a source of time-stamped information . This is both concerning and counterproductive, particularly as Annals also presents a public facing stance that apparently supports wholehearted use of clinical trial registries . Trial registers were explicitly designed as a publicly accessible source of information on prespecified outcomes in order to address selective outcome reporting. They were set up with extensive endorsement from ICMJE, journals, professional bodies, and organisations such as WHO. There is no reason for Annals not to use registers, or to dismiss their contents. With their current stance Annals are actively undermining one of the most important initiatives to improve transparency and reporting standards in the past two decades. More concerning is their preference for using protocols that are unpublished, thus reducing transparency, and to use them behind closed doors, in a manner whereby Annals’ assessments cannot be scrutinised. When COMPare initially assessed the Everson trial  published in Annals, we used the pre-specified outcomes posted by the trialists to the trial registry, because this was the only publicly accessible date-stamped source for this information. When Annals’ editors attempted to argue  that protocols are a better source of information, we asked for this protocol, which Annals reported as publicly available, but were told it was only available if we signed a legal document promising not to discuss the information . This is clearly no way to ensure best practice in trial reporting. While creating a public impression of support for improved standards, Annals are perpetuating the worst practices of the past, with a covert system unnecessarily using secret inaccessible documents behind closed doors to check for outcome switching. Worse still, this system is demonstrably failing; data from our team  and others [2, 3] have shown that Annals is repeatedly permitting undisclosed outcome switching and therefore routinely misreporting high impact clinical trials. To compound all this, Annals has sought to avoid public discussion of these shortcomings . We believe Annals’ mechanisms for addressing outcome switching are outdated and overdue for revision. We strongly encourage Annals to reconsider their position and to engage in a serious open discussion of outcome switching and their mechanisms to prevent it, on the pages of the journal. Only then can readers participate and fully understand exactly what Annals is - and is not - doing to ensure that all prespecified trial outcomes are reported.Ben Goldacre, Henry Drysdale, Kamal R. Mahtani, on behalf of the COMPare Trials Project. References: The COMPare Trials Project, www.COMPare-trials.org 2016. Fleming, Padhraig S., Despina Koletsi, Kerry Dwan, and Nikolaos Pandis. “Outcome Discrepancies and Selective Reporting: Impacting the Leading Journals?” PLoS ONE 10, no.5 (May 21, 2015): e0127495. doi:10.1371/journal.pone.0127495. Jones, Christopher W., Lukas G. Keil, Wesley C. Holland, Melissa C. Caughey, and Timothy F. Platts-Mills. “Comparison of Registered and Published Outcomes in Randomized Controlled Trials: A Systematic Review.” BMC Medicine 13 (2015): 282. doi:10.1186/s12916-015-0520-3. COMPare blog, April 5 2016: http://compare-trials.org/blog/breaches-of-publication-ethics-at-annals/  COMPare blog, January 20 2016: http://compare-trials.org/blog/where-does-annals-of-internal-medicine-stand-on-outcome-switching-a-detailed-response/  Sharing Clinical Trial Data: A Proposal From the InternationalCommittee of Medical Journal Editors, Ann Intern Med. 2016. doi:10.7326/M15-2928 Everson G et al, Sofosbuvir With Velpatasvir in Treatment-Naive Noncirrhotic Patients With Genotype 1 to 6 Hepatitis C Virus Infection, Ann Intern Med., published online first at www.annals.org on 10 November 2015. COMPare blog, April 6 2016:http://compare-trials.org/blog/another-ethical-breach-at-annals-misleading-transparency-statements-and-secret-protocols/
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only