The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

Addressing Missing Data in Clinical Trials

Thomas R. Fleming, PhD
[+] Article, Author, and Disclosure Information

From the University of Washington, Seattle, Washington.

Note: This article was delivered on 9 September 2009 as an invited presentation at the National Academy of Sciences Workshop on the Handling of Missing Data in Clinical Trials.

Acknowledgment: The author thanks Laura Guay for insights about retention procedures in the HIVNET 012 trial, Scott Emerson for his important insights shared during many discussions about this topic, and Gary Layton and Mike Oakes for providing access to their creative analysis of the pulmonary arterial hypertension data.

Grant Support: By the National Institutes of Health/National Institute of Allergy and Infectious Disease (R37 AI 29168).

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-1856.

Requests for Single Reprints: Thomas R. Fleming, PhD, Department of Biostatistics, University of Washington, Box 357232, Seattle, WA 98195-7232; e-mail, tfleming@u.washington.edu.

Author Contributions: Drafting of the article: T.R. Fleming.

Final approval of the article: T.R. Fleming.

Statistical expertise: T.R. Fleming.

Obtaining of funding: T.R. Fleming.

Ann Intern Med. 2011;154(2):113-117. doi:10.7326/0003-4819-154-2-201101180-00010
Text Size: A A A

The reliability and interpretability of results from clinical trials can be substantially reduced by missing data. Frequently used approaches to address these concerns, such as upward adjustments in sample sizes or simplistic methods for handling missing data, including last-observation-carried-forward, complete-case, or worst-case analyses, are usually inadequate. Although rational imputation methods may be useful to treat missingness after it has occurred, these methods depend on untestable assumptions. Thus, the preferred and often only satisfactory approach to addressing missing data is to prevent it. Procedures should be in place to maximize the likelihood that outcome data will be obtained at scheduled times of evaluation for all surviving patients who have not withdrawn consent. To meaningfully reduce missing data, it is important to recognize and address many factors that commonly lead to higher levels of missingness.


Grahic Jump Location
Change from baseline in 6-minute walk distance over 48 months after initiation of treatment in patients with pulmonary arterial hypertension.

At each scheduled visit, the average change from baseline in 6-minute walk distance is plotted separately for the subgroups that will and will not remain under follow-up at the time of the next scheduled visit. The numbers above the squares represent, at each scheduled visit, the number of patients in the subgroup that will remain under follow-up at the time of the next scheduled visit. Hence, these are the numbers of patients who contribute to the calculations of the means and the 95% CIs for that subgroup.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
MIssing Data and Clinical Research
Posted on February 13, 2011
Mark J. van der Laan
U. California Berkeley
Conflict of Interest: None Declared

To The Editor

We read with interest the excellent article "Addressing Missing Data in Clinical Trials" in the January 18, 2011 issue of AIM which calls attention to the problem of missing data in clinical research. Historically, patient withdrawal from clinical trials has not received the attention it deserves as a potential source of bias. Many trials have made insufficient attempts to retain patient participation in clinical trials after study treatment has been discontinued, compromising the integrity of an unbiased efficacy analysis. Many trials have made insufficient attempts to retain patient participation in clinical trials after study treatment has been discontinued, compromising the integrity of an unbiased intention-to-treat efficacy analysis. As Dr. Fleming notes, there is a need to draw a distinction between nonadherence, when patients have discontinued study article, and nonretention, when patients refuse to participate further in the observation schedule. The latter should be decreased as much as possible in order to allow for an unbiased intention to treat analysis under minimal assumptions.

However, the phenomenon of missing data will remain an issue even if Dr. Fleming's advice is strictly implemented. There will always be clinical situations in which patients will change their minds about compliance with study procedures. For example, patients with malignant may become discouraged if their own disease seems to progress and no longer wish to go through the extra time and effort it takes to remain a study patient. There are many other clinical indications that have traditionally suffered from high withdrawal rates, such as psychiatric disease and obesity.

Fortunately, there are methods being developed that improve the ability of the statistician to account for bias introduced by missing data that avoid the simplifications assumed by traditional regression methods. Techniques employing causal inference and semiparametric targeted estimation have been shown to provide unbiased and efficient estimators of the desired treatment effect under much less stringent assumptions than the assumptions current methods rely upon (1-5).

Finally, although encouraging continuing participation of all patients to continue participation to completion in clinical trials, it may become an ethical issue for the study physician who must not be seen to be too coercive in such attempts.

It is likely that the problem of missing data will remain an issue in clinical investigation which should be addressed with all the tools available to the clinical research community.

Mark van der Laan, Ph.D. Professor of Biostatistics, U.C Berkeley

Todd J. Lorenz, M.D. Clinical Research Consultant


1. M. Hernan and J.M. Robins. Causal Inference. Chapman and Hall/CRC, Boca Raton, FL, 2011.

2. J. Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, New York, 2nd edition, 2009.

3. A.A. Tsiatis. Semiparametric Theory and Missing Data. Springer, New York, 2006.

4. M.J. van der Laan and J.M. Robins. Unified Methods for Censored Longitudinal Data and Causality. Springer, New York, 2003.

5. M.J. van der Laan and S. Rose. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer, New York, 2011.

Conflict of Interest:

Dr. van der Laan has no competing interests. Dr. Lorenz is a Clinical Research Consultant for Portola Pharmaceuticals and Johnson & Johnson.

Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Related Point of Care
Topic Collections
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.