Olaide Y. Raji, PhD; Stephen W. Duffy, MSc; Olorunshola F. Agbaje, PhD; Stuart G. Baker, ScD; David C. Christiani, MD, MPH; Adrian Cassidy, PhD; John K. Field, PhD, FRCPath
Acknowledgment: The authors thank all study participants, the EUELC Consortium, Dr. Andrew J. Vickers for his useful discussion and helpful comments during the statistical data analysis and preparation of the manuscript, and Professor Anne Field for reading the manuscript as a nonexpert clinician. For members of the EUELC Consortium, see the Appendix.
Grant Support: By the Roy Castle Lung Cancer Foundation, the National Institute for Health Research Health Technology Assessment program, and the American Cancer Society, as well as grants CA74386, CA092824, and CA090578 from the National Cancer Institute, National Institutes of Health (Dr. Christiani).
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-1994.
Reproducible Research Statement:Study protocol: Available from Professor Field (e-mail, J.K.Field@liv.ac.uk). Statistical code: Available from Dr. Raji (e-mail, O.Y.Raji@liv.ac.uk). Relative utility curves are available from Dr. Baker (e-mail, firstname.lastname@example.org). Data set: LLCC and LLPC data are available from Professor Field (e-mail, J.K.Field@liv.ac.uk) on completion of a data transfer agreement. Harvard and EUELC data need separate approval for their release.
Requests for Single Reprints: John K. Field, PhD, FRCPath, Roy Castle Lung Cancer Research Programme, The University of Liverpool Cancer Research Centre, Institute of Translational Medicine, The University of Liverpool, Liverpool L3 9TA, United Kingdom; e-mail, J.K.Field@liv.ac.uk.
Current Author Addresses: Dr. Raji and Professor Field: Roy Castle Lung Cancer Research Programme, The University of Liverpool Cancer Research Centre, Institute of Translational Medicine, The University of Liverpool, Liverpool L3 9TA, United Kingdom.
Professor Duffy: Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, Charterhouse Square, London EC1M 6BQ, United Kingdom.
Dr. Agbaje: Division of Cancer Studies, Cancer Epidemiology Unit, King's College London, School of Medicine, Academic Oncology, 3rd Floor, Bermondsey Wing, Guy's Hospital, London SE1 9RT, United Kingdom.
Dr. Baker: National Cancer Institute, EPN 3131, 6130 Executive Boulevard MSC, Bethesda, MD 20892.
Dr. Christiani: Harvard School of Public Health, 665 Huntington Avenue, Boston, MA 02115.
Dr. Cassidy: GlaxoSmithKline Vaccines, Parc de la Noire Epine, Rue Fleming 20, 1300 Wavre, Belgium.
Author Contributions: Conception and design: S.W. Duffy, A. Cassidy, J.K. Field.
Analysis and interpretation of the data: O.Y. Raji, S.W. Duffy, O.F. Agbaje, S.G. Baker, D.C. Christiani, A. Cassidy, J.K. Field.
Drafting of the article: O.Y. Raji, O.F. Agbaje, A. Cassidy, J.K. Field.
Critical revision of the article for important intellectual content: O.Y. Raji, S.G. Baker, D.C. Christiani, A. Cassidy, J.K. Field.
Final approval of the article: O.Y. Raji, S.W. Duffy, O.F. Agbaje, D.C. Christiani, A. Cassidy, J.K. Field.
Provision of study materials or patients: D.C. Christiani, J.K. Field.
Statistical expertise: S.W. Duffy, S.G. Baker.
Obtaining of funding: D.C. Christiani, J.K. Field.
Administrative, technical, or logistic support: D.C. Christiani, J.K. Field.
Collection and assembly of data: O.Y. Raji, A. Cassidy, J.K. Field.
Raji O., Duffy S., Agbaje O., Baker S., Christiani D., Cassidy A., Field J.; Predictive Accuracy of the Liverpool Lung Project Risk Model for Stratifying Patients for Computed Tomography Screening for Lung Cancer: A Case–Control and Cohort Validation Study. Ann Intern Med. 2012;157:242-250. doi: 10.7326/0003-4819-157-4-201208210-00004
Download citation file:
Published: Ann Intern Med. 2012;157(4):242-250.
External validation of existing lung cancer risk prediction models is limited. Using such models in clinical practice to guide the referral of patients for computed tomography (CT) screening for lung cancer depends on external validation and evidence of predicted clinical benefit.
To evaluate the discrimination of the Liverpool Lung Project (LLP) risk model and demonstrate its predicted benefit for stratifying patients for CT screening by using data from 3 independent studies from Europe and North America.
Case–control and prospective cohort study.
Europe and North America.
Participants in the European Early Lung Cancer (EUELC) and Harvard case–control studies and the LLP population-based prospective cohort (LLPC) study.
5-year absolute risks for lung cancer predicted by the LLP model.
The LLP risk model had good discrimination in both the Harvard (area under the receiver-operating characteristic curve [AUC], 0.76 [95% CI, 0.75 to 0.78]) and the LLPC (AUC, 0.82 [CI, 0.80 to 0.85]) studies and modest discrimination in the EUELC (AUC, 0.67 [CI, 0.64 to 0.69]) study. The decision utility analysis, which incorporates the harms and benefit of using a risk model to make clinical decisions, indicates that the LLP risk model performed better than smoking duration or family history alone in stratifying high-risk patients for lung cancer CT screening.
The model cannot assess whether including other risk factors, such as lung function or genetic markers, would improve accuracy. Lack of information on asbestos exposure in the LLPC limited the ability to validate the complete LLP risk model.
Validation of the LLP risk model in 3 independent external data sets demonstrated good discrimination and evidence of predicted benefits for stratifying patients for lung cancer CT screening. Further studies are needed to prospectively evaluate model performance and evaluate the optimal population risk thresholds for initiating lung cancer screening.
Roy Castle Lung Cancer Foundation.
Robert P Young, MD, Raewyn J Hopkins, MD, David E Midthun, MD
Drs. Young and Hopkins -Schools of Biological Science and Faculty of Medical and Health Sciences, Univ of Auckland, New Zealand. Dr. Midthun - Mayo Clinic, Rochester, Minnesota, USA.
September 12, 2012
Conflict of Interest:
Financial/nonfinancial disclosures: RPY, and the funding of his research, has been supported by grants from the University of Auckland, Health Research Council of New Zealand and Synergenz BioSciences Ltd. DEM, has received payment for preparation of chapters on lung cancer screening for the American College of Physicians and royalties for a chapter on lung cancer in Up-to-Date.
Predictive Accuracy of the Liverpool Lung Project (LLP) Risk Model for Stratifying Patients for Computed Tomography Screening for Lung Cancer
TO THE EDITOR:
We agree with Raji et al.(1), that the selection of current and former smokers for computed tomographic (CT) screening for lung cancer should target those at greatest risk “to maximize the benefit-harm ratio” (2). While we concur that a multivariate approach to risk assessment is the best way to achieve this (3), we question whether the validated LLP model maximizes this benefit-harm ratio as suggested.
Although the LLP multivariate model performs better than lung cancer risk models using age and smoking history alone (1), it is less clear that this superior performance translates into improved CT screening outcomes. In a recent opinion piece by Bach and Gould (4), it was strongly argued that screening should be limited to those at greatest risk so that the greatest number of deaths from lung cancer can be averted per person screened. Using the results of the National Lung Screening Trial (NLST), Bach et al. showed that the absolute number of deaths averted by screening is maximized when lung cancer detection rate (or death rate) is maximized. The study by Raji and colleagues (1) does not demonstrate a superior detection rate using the LLP model which the authors themselves have estimated to be only 1.0-1.5 fold that from using the NLST criteria (5). If the LLP model does not substantially increase the detection rate of lung cancer (1), it is difficult to see how it will maximize the benefit-harm ratio (4).
Given the LLP model uses more variables than the Bach model (also externally validated (4)), and achieves a higher area-under-the-curve performance characteristic (AUC), why then might the detection rate be no different to the NLST criteria that uses age and smoking history alone? It may be because the arbitrary cut off of 5% used for selection based on the LLP model is too low. Only validation in a CT screening study will clarify this issue. We have recently shown, using the Pittsburgh CT Screening Study, that when eligible smokers were stratified according to the presence of COPD, the lung cancer detection rate was 5 fold greater than in those without COPD (2,3). It is therefore possible that the absence of COPD in the LLP model may reduce its utility in distinguishing those most at risk of lung cancer among older heavy smokers. The role of “COPD-related” risk factors in improving lung cancer detection rate is currently under investigation in the NLST (2).
1. Raji OY, Duffy SW, Agbaje OF, Baker SG, Christianni DC, et al. Predictive accuracy of the Liverpool Lung Project risk model for stratifying patients for computed tomography screening for lung cancer: A case-control and cohort validation study. Ann Int Med 2012; 157: 242-250.
2. Young RP, Hopkins RJ.CT screening for lung cancer. Thorax 2012; 67: 650-651.
3. Young RP, Hopkins RJ. Diagnosing COPD and targeted lung cancer screening. Eur Respir J 2012; doi:10.1183/09031936.0070012.
4. Bach PB, Gould MK. When the average applies to no one: personalized decision making about potential benefits of lung cancer screening. Ann Int Med 2012, August 14.
5. Baldwin DR, Duffy SW, Wald NJ, Page R, Hansell DM, Field JK, et al. UK Lung Screen (UKLS) nodule management protocol: modeling of a single screen randomized control trial of low-dose CT screening for lung cancer. Thorax 2011; 66: 308-313.
John K. Field (PhD, FRCPath)1 ,Olaide Y. Raji (PhD)1, Stephen W. Duffy (MSc)2,
1.Roy Castle Lung Cancer Res Prgm, Univ of Liverpool Cancer Res Ctr, Inst. of Trsltnl Med, Univ of Liverpool. 2.Wolfson Inst of Preventive Med, Barts & London Sch of Med, Queen Mary Univ of Lndn
October 12, 2012
We thank Dr. Young and colleague for their interest and useful comments on our recent article (1). The LLP risk model was demonstrated to perform better than models based on age and smoking history alone. The superior net benefit shown in three independent studies suggested a predicted clinical utility in terms of screening decision at relevant thresholds; however this is a retrospective performance assessment and the actual effect can only be shown with prospective CT screening trials.
The detection rate of 1.5% reported in Baldwin et al 2011 (2) was a conservative estimate used to power the UKLS trial, and quite possibly be an underestimation. The question as to whether the LLP risk model achieves the objective of identifying a ‘high-risk‘ population, will only be proven when the UKLS CT screening trial is completed, with the necessary follow-up and the cost effectiveness of the approach has been demonstrated. Currently the academic community is still in the ‘modelling’ mindset and these discussions will continue on the clinical usefulness of lung cancer risk models, in particular, the LLP risk model until the UKLS trial has been completed.
We agree with Dr. Young and colleagues that COPD is an important independent risk factor for lung cancer, with a two to five fold increase in lung cancer reported among smokers. This strong independent relationship suggests a possible overlap in the biology of the two diseases and possibly sharing similar risk factor in smoking. Whilst, an independent association was found between COPD and lung cancer in the LLP dataset, it was not significant in the multivariable model (3). The lack of significant association of COPD may be due to reporting bias as diagnosis of COPD may have been under reported in the study area.
The LLP risk model has the advantage of simplicity of use and interpretation, and identifies subjects who are at high risk due to risk factors, other than smoking. The model would be most useful in population-based screening, where there is a low COPD prevalence.
1. Raji YR, Duffy SW, Agbaje OF, Baker SG, Christiani DC, Cassidy A, et al. Predictive Accuracy of the Liverpool Lung Project Risk Model for Stratifying Patients for Computed Tomography Screening for Lung Cancer. A Case–Control and Cohort Validation Study. Annals of Internal Medicine. 2012;21:128-38.
2. Baldwin DR, Duffy SW, Wald NJ, Page R, Hansell DM, Field JK. UK Lung Screen (UKLS) nodule management protocol: modelling of a single screen randomised controlled trial of low-dose CT screening for lung cancer. Thorax. 2011 Apr;66(4):308-13.
3. Cassidy A, Myles J, van-Tongeren M, Page R, Liloglou T, Duffy S, et al. The LLP risk model: an individual risk prediction model for lung cancer. Br J Cancer. 2008 Epub 2007 Dec 18;98((2)):270-6.
to gain full access to the content and tools.
Learn more about subscription options.
Register Now for a free account.
Hematology/Oncology, Pulmonary/Critical Care, Lung Cancer, Cancer Screening/Prevention, Prevention/Screening.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only