Skip Navigation
Menu
Editorials |16 October 2018

Inappropriate Statistical Analysis and Reporting in Medical Research: Perverse Incentives and Institutional Solutions

A. Russell Localio, PhD; Catharine B. Stack, PhD; Anne R. Meibohm, PhD; Eric A. Ross, PhD; Eliseo Guallar, MD, DrPH; John B. Wong, MD; John E. Cornell, PhD; Michael E. Griswold, PhD; Steven N. Goodman, MD, MHS, PhD

Article, Author, and Disclosure Information
Author, Article, and Disclosure Information
This article was published at Annals.org on 9 October 2018.
  • University of Pennsylvania, Philadelphia, Pennsylvania (A.R.L.)
    American College of Physicians, Philadelphia, Pennsylvania (C.B.S., A.R.M.)
    Fox Chase Cancer Center, Philadelphia, Pennsylvania (E.A.R.)
    Johns Hopkins University, Baltimore, Maryland (E.G.)
    Tufts University School of Medicine, Boston, Massachusetts (J.B.W.)
    University of Texas Health Science Center, San Antonio, Texas (J.E.C.)
    University of Mississippi Medical Center, Jackson, Mississippi (M.E.G.)
    Stanford University School of Medicine, Stanford, California (S.N.G.)

    Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M18-2516.

    Corresponding Author: A. Russell Localio, PhD, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, 617 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104; e-mail, rlocalio@pennmedicine.upenn.edu.

    Current Author Addresses: Dr. Localio: Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, 617 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104.

    Drs. Stack and Meibohm: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.

    Dr. Ross: Fox Chase Cancer Center, 333 Cottman Avenue, Philadelphia, PA 19111.

    Dr. Guallar: Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins Medical Institutions, Room 2-645, Baltimore, MD 21205.

    Dr. Wong: Tufts Medical Center, 800 Washington Street, #302, Boston, MA 02111.

    Dr. Cornell: 14546 Indian Woods, San Antonio, TX 78249.

    Dr. Griswold: Department of Data Science, John D. Bower School of Population Health, University of Mississippi Medical Center, 2500 North State Street, Jackson, MS 39216.

    Dr. Goodman: Stanford University School of Medicine, 259 Campus Drive, T265 Redwood Building/Health Research and Policy, Stanford, CA 94305.

×
FULL TEXT
Wang and colleagues (1) present a sobering report of a national survey of nearly 400 consulting statisticians about requests from investigators to engage in inappropriate statistical practices. Framed as an exploration of bioethical issues, the report implicitly adopts Doug Altman's mantra: “Misuse of statistics is unethical” (2). Although the survey did not ask statisticians whether they fulfilled these requests, the inappropriate methods described in this report are still used in the published literature, and thus contribute to the problem of nonreproducible research.
Practices like these are extraordinarily difficult to detect in published work; identification takes either unusual transparency or a time-consuming re-examination of the original research methods and data. We cannot determine whether these requests arise largely from researchers' inexperience or their response to academic and professional incentives that reward impressive-looking results in higher-profile publications.
Educational and structural changes in biostatistics cannot counteract these incentives completely, but they can reduce the role of inexperience. In responses to Nature's online survey of 1576 researchers about ways to “boost research reproducibility,” the top choice was “better understanding of statistics,” followed by “better mentoring/supervision,” “more robust designs,” and “better teaching” (3). Attaining these goals requires an altered structure for biostatistics education and collaboration.
Introductory courses and textbooks in biostatistics for clinician-investigators often focus on formulae and software. However, even more powerful, informal curricula of discipline-specific statistical practices are available (not all correct), gleaned from published literature, journal clubs, research presentations, and grant proposals. Collectively, these sources...
FULL TEXT
PDF
Not Available
Citations
Citation

Localio AR, Stack CB, Meibohm AR, Ross EA, Guallar E, Wong JB, et al. Inappropriate Statistical Analysis and Reporting in Medical Research: Perverse Incentives and Institutional Solutions. Ann Intern Med. [Epub ahead of print 9 October 2018]169:577–578. doi: 10.7326/M18-2516

Download citation file:

© 2019

×
Permissions

Published: Ann Intern Med. 2018;169(8):577-578.

DOI: 10.7326/M18-2516

Published at www.annals.org on 9 October 2018

©
2018 American College of Physicians
1 Citations

Related Articles
The Importance of Reporting Biases in Patient Care: Can We Trust the Evidence From Either Individual Studies or Systematic Reviews?
Annals of Internal Medicine; 169 (6): 413-414
The Value-Based Payment Modifier: Program Outcomes and Implications for Disparities
Annals of Internal Medicine; 168 (4): 255-265
U.S. Immigration Policy and American Medical Research: The Scientific Contributions of Foreign Medical Graduates
Annals of Internal Medicine; 167 (8): 584-586
Raising the Bar in Attribution
Annals of Internal Medicine; 167 (6): 434-435
View MoreView Less

Related Topics
Education and Training
Ethics
Hematology/Oncology
Neurology

Education and Training, Ethics, Hematology/Oncology, Neurology.

PubMed Articles
Differential contributions of striatal dopamine D1 and D2 receptors to component processes of value-based decision making.
Neuropsychopharmacology 2019.
Design, synthesis and activity of novel 2,6-disubstituted purine derivatives, potential small molecule inhibitors of signal transducer and activator of transcription 3.
Eur J Med Chem 2019;
View More

Results provided by: PubMed

CONTINUE WITH LIMITED ACCESS
Annals of Internal Medicine
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
acp link acp
silverchair link silverchair

Copyright © 2019 American College of Physicians. All Rights Reserved.

Print ISSN: 0003-4819 | Online ISSN: 1539-3704

Privacy Policy

|

Conditions of Use

This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy. | Accept
×

You need a subscription to this content to use this feature.

×
PDF Downloads Require Access to the Full Article.
Annals of Internal Medicine
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×
Access to this Free Content Requires Users to be Registered and Logged In. Please Choose One of the Following Options
Annals of Internal Medicine
To receive access to the full text of freely available articles, alerts, and more. You will be directed to acponline.org to complete your registration.
×