Katherine S. Panageas, DrPH; Deborah Schrag, MD, MPH; Elyn Riedel, MA; Peter B. Bach, MD, MAPP; Colin B. Begg, PhD
Acknowledgments: The authors thank 2 anonymous reviewers for help with the article. They also thank the groups responsible for the creation and dissemination of the linked database, including the Applied Research Branch, Division of Cancer Control and Population Sciences, National Cancer Institute; the Office of Strategic Planning and the Office of Informational Services, Centers for Medicare & Medicaid Services; Information Management Services; and the Surveillance, Epidemiology, and End Results tumor registries.
Grant Support: In part by grants from the National Cancer Institute (CA83950 [Dr. Schrag], CA90226 [Dr. Bach], and CA08748 [Dr. Begg]).
Potential Financial Conflicts of Interest: None disclosed.
Requests for Single Reprints: Colin B. Begg, PhD, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street (3rd Floor), New York, NY 10021; e-mail, email@example.com.
Current Author Addresses: Drs. Panageas, Schrag, Bach, and Begg and Ms. Riedel: Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street (3rd Floor), New York, NY 10021.
Author Contributions: Conception and design: K.S. Panageas, D. Schrag, E. Riedel, C.B. Begg.
Analysis and interpretation of the data: K.S. Panageas, D. Schrag, E. Riedel, C.B. Begg.
Drafting of the article: K.S. Panageas, D. Schrag, C.B. Begg.
Critical revision of the article for important intellectual content: K.S. Panageas, D. Schrag, P.B. Bach, C.B. Begg.
Final approval of the article: K.S. Panageas, P.B. Bach, C.B. Begg.
Provision of study materials or patients: D. Schrag, C.B. Begg.
Statistical expertise: K.S. Panageas, E. Riedel, C.B. Begg.
Obtaining of funding: C.B. Begg.
Administrative, technical, or logistic support: C.B. Begg.
Collection and assembly of data: D. Schrag, C.B. Begg.
Panageas K., Schrag D., Riedel E., Bach P., Begg C.; The Effect of Clustering of Outcomes on the Association of Procedure Volume and Surgical Outcomes. Ann Intern Med. 2003;139:658-665. doi: 10.7326/0003-4819-139-8-200310210-00009
Download citation file:
Published: Ann Intern Med. 2003;139(8):658-665.
In an attempt to evaluate the degree to which the choice of provider affects outcomes of major medical procedures, numerous investigators have used procedure volume as a proxy for expertise and have conducted studies correlating volume with outcomes. However, volume is a crude, easily calculated measure, and its use may overlook large variations in quality among providers that are independent of the number of procedures performed. If there are such large variations in outcomes among providers, the outcomes of patients treated by the same provider are necessarily correlated, or “clustered.” Clustering of this nature invalidates conventional statistical analyses. Patients treated at the same hospital or by the same surgeon may be more likely to experience similar outcomes if surgical technique or supportive care practices vary among providers and if these factors affect outcomes. It is well established that statistical methods must correct for the effect of clustering of this nature if it exists (1-3). In general, correction for clustering attenuates the statistical significance of observed trends.
Maren K Olsen
Duke University Medical Center
March 22, 2005
Population-Average vs. Cluster-Specific Estimates
To the Editor:
Panageas and colleagues  analyzed volume-outcome trends using three different methods: standard logistic regression, a logistic model with confidence intervals adjusted for clustering using the generalized estimating equations (GEE) method, and a random-effects logistic model. Tables 1, 2, and 3 present a side-by-side comparison of results from these three methods whereby the importance of adjusting for clustering is illustrated.
From a methodological standpoint, the reported results from the random-effects models are surprising. The odds ratio from the GEE method represents a population-average (PA) estimate; the odds ratio from the random-effects models represents a cluster-specific (CS) estimate (e.g., see Localio et al ). When there is clustering, the estimates from a random-effects model are expected to be larger (i.e., farther from the null value) than GEE estimates; the discrepancy is dependent on the variance of the random effect (sc2). For a logistic model, the PA effect is related to the CS effect using the following formula [3, p. 136]: bPA ~= bcs(1 + 0.346sc2)-1/2
In Table 2, however, the estimates from the random effects models are smaller than the estimates from GEE (1.46 vs. 1.58 and 1.88 vs. 2.32). Similarly, in Table 3, the random-effects model estimate for the abdominoperineal resection outcome is smaller than the GEE estimate (1.09 vs. 1.21).
It is not clear why the random-effects model estimates are smaller than the GEE estimates. One possibility is computational error. The authors used gllamm6 in Stata to fit the random-effects models. Given the evolving state of computational methods, the authors may want to consider other procedures (e.g., gllamm in Stata Version 8 or proc nlmixed in SAS) to verify the random-effects model results. It would also have been helpful if an estimate of sc2 or the ICC had been included.
In the discussion section, the authors note the "disconcertingly large differences in the results" and further state that, "both statistical methods endeavor to estimate the same effect, the odds ratio of volume on outcome, and the discrepancies in estimates must reflect their different technical formulations." These methods do not estimate the same effect, and this affects the interpretation of the estimates . The random-effects model is a conditional method, and the estimated odds ratio is conditional upon cluster (e.g., surgeon). In contrast, GEE is an unconditional method, and the estimated odds ratio is the overall effect averaged across clusters. As an illustrative paper, the authors should have drawn more careful distinctions between these estimates and their interpretations.
Maren K. Olsen, PhD Duke University Medical Center Durham, NC 27705
John S. Preisser, PhD University of North Carolina, School of Public Health Chapel Hill, NC 27599
1. Panageas KS, Schrag D, Riedel E, Bach PB, Begg CB. The effect of clustering on the association of procedure volume and surgical outcomes. Ann Intern Med 2003;139(8):648-665.
2. Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an overview. Ann Intern Med. 2001;135:112- 23.
3. Diggle PG, Heagerty P, Liang KY, Zeger SL. 2002. Analysis of Longitudinal Data, 2nd edition. Oxford University Press, New York.
to gain full access to the content and tools.
Learn more about subscription options.
Register Now for a free account.
Gastroenterology/Hepatology, Hematology/Oncology, Hospital Medicine, Healthcare Delivery and Policy, Prostate Cancer.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only