The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research |

When To Base Clinical Policies on Observational Versus Randomized Trial Data

John Hornberger, MD, MS; and Elizabeth Wrone, MD
[+] Article, Author, and Disclosure Information

From the Stanford University School of Medicine, Stanford, California. Note: This article is one of a series of articles comprising an Annals of Internal Medicine supplement entitled “Measuring Quality, Outcomes, and Cost of Care Using Large Databases: The Sixth Regenstrief Conference.” To see a complete list of the articles included in this supplement, please view its Table of Contents. Acknowledgments: The authors thank Siu Hui, Linda McCann, Philip Lavori, Mary Anne Rodgers, and two anonymous referees for their comments on the manuscript. Requests for Reprints: John Hornberger, MD, MS, Department of Health Research and Policy, Stanford University School of Medicine, HAP Redwood Building, Room T254A, Stanford, CA 94305-5092. Current Author Addresses: Dr. Hornberger: Department of Health Research and Policy, Stanford University School of Medicine, HAP Redwood Building, Room T254A, Stanford, CA 94305-5092. Dr. Wrone: Department of Medicine, Stanford University School of Medicine, HAP Redwood Building, Room T254B, Stanford, CA 94305-5092.

Copyright ©2004 by the American College of Physicians

Ann Intern Med. 1997;127(8_Part_2):697-703. doi:10.7326/0003-4819-127-8_Part_2-199710151-00053
Text Size: A A A

Physicians must decide when the evidence is sufficient to adopt a new clinical policy. Analysis of large clinical and administrative databases is becoming an important source of evidence for changing clinical policies. Because such analysis cannot control for the effects of all potential confounding variables, physicians risk drawing the wrong conclusion about the cause-and-effect relation between a change in clinical policy and outcomes. Randomized studies offer protection against drawing a conclusion that would lead to adoption of an inferior policy. However, a randomized study may be difficult to justify because of the extra costs of collecting data for a randomized study and concerns that a study will not directly benefit the patients enrolled in the study. This article reviews the advantages and disadvantages of basing clinical policy on analysis of large databases compared with conducting a randomized study. A technique is described and illustrated for accessing the potential costs and benefits of conducting such a study. This type of analysis formed the basis for a physician-managed health care organization deciding to sponsor a randomized study among patients with end-stage renal disease as part of a quality-improvement initiative.


Grahic Jump Location
Figure 2.
Plot of the probability distribution of the relative risk for death for the prior and posterior distributions with different sample sizes.

The prior distribution has a β distribution. These analyses assume that the estimated relative risk of the trial is 0.70. With larger sample sizes (for example, 50 and 200 patients per group), the mean of the posterior distribution shifts closer to 0.70 and the SD shrinks.

Grahic Jump Location
Grahic Jump Location
Figure 1.
Expected cost–benefit difference and loss. Top.Middle.Bottom.

The expected cost–benefit difference per patient if high-dose folic acid is recommended instead of standard-dose folic acid for different values of the relative risk for death of the two treatments. The loss per patient through recommending standard-dose folic acid. The loss per patient through recommending high-dose folic acid.

Grahic Jump Location
Grahic Jump Location
Figure 3.
Sensitivity analyses.Top.Bottom.

Net value for conducting a trial compared with waiting until another organization publishes its trial results, as a function of the prior distribution. Net savings at different means of the prior distribution where SD remained equal to 0.2. Net savings at different SDs of the prior distribution where the mean remained equal to 1.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Related Point of Care
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.