The full content of Annals is available to subscribers

Subscribe/Learn More  >
Original Research |

The Effect of Clustering of Outcomes on the Association of Procedure Volume and Surgical Outcomes

Katherine S. Panageas, DrPH; Deborah Schrag, MD, MPH; Elyn Riedel, MA; Peter B. Bach, MD, MAPP; and Colin B. Begg, PhD
[+] Article, Author, and Disclosure Information

From Memorial Sloan-Kettering Cancer Center, New York, New York.

Acknowledgments: The authors thank 2 anonymous reviewers for help with the article. They also thank the groups responsible for the creation and dissemination of the linked database, including the Applied Research Branch, Division of Cancer Control and Population Sciences, National Cancer Institute; the Office of Strategic Planning and the Office of Informational Services, Centers for Medicare & Medicaid Services; Information Management Services; and the Surveillance, Epidemiology, and End Results tumor registries.

Grant Support: In part by grants from the National Cancer Institute (CA83950 [Dr. Schrag], CA90226 [Dr. Bach], and CA08748 [Dr. Begg]).

Potential Financial Conflicts of Interest: None disclosed.

Requests for Single Reprints: Colin B. Begg, PhD, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street (3rd Floor), New York, NY 10021; e-mail, beggc@mskcc.org.

Current Author Addresses: Drs. Panageas, Schrag, Bach, and Begg and Ms. Riedel: Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street (3rd Floor), New York, NY 10021.

Author Contributions: Conception and design: K.S. Panageas, D. Schrag, E. Riedel, C.B. Begg.

Analysis and interpretation of the data: K.S. Panageas, D. Schrag, E. Riedel, C.B. Begg.

Drafting of the article: K.S. Panageas, D. Schrag, C.B. Begg.

Critical revision of the article for important intellectual content: K.S. Panageas, D. Schrag, P.B. Bach, C.B. Begg.

Final approval of the article: K.S. Panageas, P.B. Bach, C.B. Begg.

Provision of study materials or patients: D. Schrag, C.B. Begg.

Statistical expertise: K.S. Panageas, E. Riedel, C.B. Begg.

Obtaining of funding: C.B. Begg.

Administrative, technical, or logistic support: C.B. Begg.

Collection and assembly of data: D. Schrag, C.B. Begg.

Ann Intern Med. 2003;139(8):658-665. doi:10.7326/0003-4819-139-8-200310210-00009
Text Size: A A A

Results are shown in Tables 1, 2, and 3 for colon, prostate, and rectal cancer, respectively. For each clinical outcome, the results are characterized by the odds ratio of the outcome per 100-unit change in surgeon volume (per 10-unit change for rectal cancer).

First Page Preview

View Large
First page PDF preview


Grahic Jump Location
Figure 1.
Two superimposed histograms of the numbers of surgeons with observed and expected 2-year mortality rates after colon cancer surgery.
Grahic Jump Location
Grahic Jump Location
Figure 2.
Two superimposed histograms of the numbers of surgeons with observed and expected ostomy rates after colon cancer surgery.
Grahic Jump Location
Grahic Jump Location
Figure 3.
Two superimposed histograms of the numbers of surgeons with observed and expected rates of late urinary complication after prostate cancer surgery.
Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Population-Average vs. Cluster-Specific Estimates
Posted on March 22, 2005
Maren K Olsen
Duke University Medical Center
Conflict of Interest: None Declared

To the Editor:

Panageas and colleagues [1] analyzed volume-outcome trends using three different methods: standard logistic regression, a logistic model with confidence intervals adjusted for clustering using the generalized estimating equations (GEE) method, and a random-effects logistic model. Tables 1, 2, and 3 present a side-by-side comparison of results from these three methods whereby the importance of adjusting for clustering is illustrated.

From a methodological standpoint, the reported results from the random-effects models are surprising. The odds ratio from the GEE method represents a population-average (PA) estimate; the odds ratio from the random-effects models represents a cluster-specific (CS) estimate (e.g., see Localio et al [2]). When there is clustering, the estimates from a random-effects model are expected to be larger (i.e., farther from the null value) than GEE estimates; the discrepancy is dependent on the variance of the random effect (sc2). For a logistic model, the PA effect is related to the CS effect using the following formula [3, p. 136]: bPA ~= bcs(1 + 0.346sc2)-1/2

In Table 2, however, the estimates from the random effects models are smaller than the estimates from GEE (1.46 vs. 1.58 and 1.88 vs. 2.32). Similarly, in Table 3, the random-effects model estimate for the abdominoperineal resection outcome is smaller than the GEE estimate (1.09 vs. 1.21).

It is not clear why the random-effects model estimates are smaller than the GEE estimates. One possibility is computational error. The authors used gllamm6 in Stata to fit the random-effects models. Given the evolving state of computational methods, the authors may want to consider other procedures (e.g., gllamm in Stata Version 8 or proc nlmixed in SAS) to verify the random-effects model results. It would also have been helpful if an estimate of sc2 or the ICC had been included.

In the discussion section, the authors note the "disconcertingly large differences in the results" and further state that, "both statistical methods endeavor to estimate the same effect, the odds ratio of volume on outcome, and the discrepancies in estimates must reflect their different technical formulations." These methods do not estimate the same effect, and this affects the interpretation of the estimates [2]. The random-effects model is a conditional method, and the estimated odds ratio is conditional upon cluster (e.g., surgeon). In contrast, GEE is an unconditional method, and the estimated odds ratio is the overall effect averaged across clusters. As an illustrative paper, the authors should have drawn more careful distinctions between these estimates and their interpretations.

Maren K. Olsen, PhD Duke University Medical Center Durham, NC 27705

John S. Preisser, PhD University of North Carolina, School of Public Health Chapel Hill, NC 27599


1. Panageas KS, Schrag D, Riedel E, Bach PB, Begg CB. The effect of clustering on the association of procedure volume and surgical outcomes. Ann Intern Med 2003;139(8):648-665.

2. Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an overview. Ann Intern Med. 2001;135:112- 23.

3. Diggle PG, Heagerty P, Liang KY, Zeger SL. 2002. Analysis of Longitudinal Data, 2nd edition. Oxford University Press, New York.

Conflict of Interest:

None declared

Submit a Comment/Letter

Summary for Patients

Choice of Statistical Analysis Can Change the Results of Studies of the Relationship between Hospital and Surgeon Volume and Outcomes of Cancer Surgery

The summary below is from the full report titled “The Effect of Clustering of Outcomes on the Association of Procedure Volume and Surgical Outcomes.” It is in the 21 October 2003 issue of Annals of Internal Medicine (volume 139, pages 658-665). The authors are K.S. Panageas, D. Schrag, E. Riedel, P.B. Bach, and C.B. Begg.


Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Related Point of Care
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.