The full content of Annals is available to subscribers

Subscribe/Learn More  >
Original Research |

Multicenter Validation of the Diagnostic Accuracy of a Blood-Based Gene Expression Test for Assessing Obstructive Coronary Artery Disease in Nondiabetic Patients

Steven Rosenberg, PhD; Michael R. Elashoff, PhD; Philip Beineke, BS; Susan E. Daniels, PhD; James A. Wingrove, PhD; Whittemore G. Tingley, MD, PhD; Philip T. Sager, MD; Amy J. Sehnert, MD; May Yau, MS; William E. Kraus, MD; L. Kristin Newby, MD; Robert S. Schwartz, MD; Szilard Voros, MD; Stephen G. Ellis, MD; Naeem Tahirkheli, MD; Ron Waksman, MD; John McPherson, MD; Alexandra Lansky, MD; Mary E. Winn, BS; Nicholas J. Schork, PhD; Eric J. Topol, MD, for the PREDICT (Personalized Risk Evaluation and Diagnosis in the Coronary Tree) Investigators
[+] Article, Author, and Disclosure Information

For a list of the PREDICT investigators, see Appendix 1.

From CardioDx, Palo Alto, California; Duke University School of Medicine, Durham, North Carolina; Minneapolis Heart Institute and Foundation, Minneapolis, Minnesota; Piedmont Heart Institute, Atlanta, Georgia; Cleveland Clinic Foundation, Cleveland, Ohio; Oklahoma Cardiovascular Research Group, Oklahoma City, Oklahoma; Cardiovascular Research Institute, Medstar Research Institute, Washington, DC; Vanderbilt Heart and Vascular Institute, Nashville, Tennessee; Cardiovascular Research Foundation, New York, New York; and Scripps Translational Science Institute, La Jolla, California.

Acknowledgment: The authors thank Michael Walker, Richard Lawn, and Fred Cohen for their helpful suggestions on the manuscript. They also thank all the patients who provided samples for the PREDICT study, as well as the study site research coordinators and those who contributed to patient recruitment, clinical data acquisition and verification, validation study experimental work, and data analysis.

Grant Support: By CardioDx and also in part by a Scripps Translational Science Institute Clinical Translational Science Award (NIHUL1RR025774) (Ms. Winn and Drs. Schork and Topol).

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-1332.

Reproducible Research Statement:Study protocol: Available at ClinicalTrials.gov (registration number: NCT00500617). Statistical code: Available to appropriate investigators from Dr. Elashoff (e-mail, melashoff@cardiodx.com). Data set: Available from Dr. Rosenberg (e-mail, srosenberg@cardiodx.com) to appropriate investigators with a defined analysis plan.

Requests for Single Reprints: Eric J. Topol, MD, Scripps Translational Science Institute, 3344 North Torrey Pines Court, La Jolla, CA 92037; e-mail, etopol@scripps.edu.

Current Author Addresses: Drs. Rosenberg, Elashoff, Daniels, and Wingrove; Mr. Beineke; and Ms. Yau: CardioDx, 2500 Faber Place, Palo Alto, CA 94303.

Dr. Tingley: Division of Cardiology, University of California, San Francisco 505 Parnassus Avenue, San Francisco, CA 94143.

Dr. Sager: Gilead Sciences, 3172 Porter Drive, Palo Alto, CA 94304.

Dr. Sehnert: Artemis Health, 1531 Industrial Road, San Carlos, CA 94070.

Dr. Kraus: Duke Center for Living, 3475 Erwin Road, Box 3022, Room 254, Aesthetics Building, Durham, NC 27705.

Dr. Newby: Duke Clinical Research Institute, Box 17969, Durham, NC 27715-7969.

Dr. Schwartz: Minneapolis Heart Institute Foundation, Abbott Northwestern Hospital, 920 East 28th Street, Suite 620, Minneapolis, MN 55407.

Dr. Voros: Piedmont Heart Institute, 95 Collier Road Northwest, Suite 2035, Atlanta, GA 30309.

Dr. Ellis: The Cleveland Clinic, 9500 Euclid Avenue, F25, Cleveland, OH 44195.

Dr. Tahirkheli: 4221 South Western Avenue, Suite 4000, Oklahoma City, OK 73109.

Dr. Waksman: Cardiovascular Research Institute, Medstar Research Institute, Washington Hospital Center, 110 Irving Street Northwest, Suite 6B-5, Washington, DC 20010.

Dr. McPherson: 1215 21st Avenue South, MCE, 5th Floor, South Tower, Nashville, TN 37232.

Dr. Lansky: Yale University School of Medicine, Fitkin Pavilion, New Haven, CT 06520.

Ms. Winn and Drs. Schork and Topol: Scripps Translational Science Institute, 3344 North Torrey Pines Court, La Jolla, CA 92037.

Author Contributions: Conception and design: S. Rosenberg, M.R. Elashoff, P. Beineke, S.E. Daniels, J.A. Wingrove, W.G. Tingley, P.T. Sager, A.J. Sehnert, M. Yau, L.K. Newby, S. Voros, A. Lansky, E.J. Topol.

Analysis and interpretation of the data: S. Rosenberg, M.R. Elashoff, S.E. Daniels, J.A. Wingrove, W.G. Tingley, P.T. Sager, W.E. Kraus, L.K. Newby, S. Voros, A. Lansky, N.J. Schork, M.E. Winn, E.J. Topol.

Drafting of the article: S. Rosenberg, M.R. Elashoff, S.E. Daniels, J.A. Wingrove, W.G. Tingley, A.J. Sehnert, R.S. Schwartz, N. Tahirkheli, A. Lansky, N.J. Schork, E.J. Topol.

Critical revision of the article for important intellectual content: S. Rosenberg, M.R. Elashoff, S.E. Daniels, J.A. Wingrove, P.T. Sager, W.E. Kraus, L.K. Newby, S. Voros, S. Ellis, N. Tahirkheli, R. Waksman, A. Lansky, E.J. Topol.

Final approval of the article: S. Rosenberg, M.R. Elashoff, S.E. Daniels, J.A. Wingrove, W.G. Tingley, P.T. Sager, M. Yau, W.E. Kraus, L.K. Newby, R.S. Schwartz, S. Voros, S. Ellis, N. Tahirkheli, R. Waksman, A. Lansky, N.J. Schork, M.E. Winn, E.J. Topol.

Provision of study materials or patients: L.K. Newby, S. Voros, S. Ellis, N. Tahirkheli, J. McPherson.

Statistical expertise: M.R. Elashoff, P. Beineke, A. Lansky, N.J. Schork.

Obtaining of funding: P.T. Sager, A. Lansky.

Administrative, technical, or logistic support: S.E. Daniels, J.A. Wingrove, P.T. Sager, M. Yau, A. Lansky, E.J. Topol.

Collection and assembly of data: S. Rosenberg, S.E. Daniels, J.A. Wingrove, W.G. Tingley, P.T. Sager, A.J. Sehnert, M. Yau, W.E. Kraus, R.S. Schwartz, S. Voros, N. Tahirkheli, J. McPherson, A. Lansky, E.J. Topol.

Ann Intern Med. 2010;153(7):425-434. doi:10.7326/0003-4819-153-7-201010050-00005
Text Size: A A A

Background: Diagnosing obstructive coronary artery disease (CAD) in at-risk patients can be challenging and typically requires both noninvasive imaging methods and coronary angiography, the gold standard. Previous studies have suggested that peripheral blood gene expression can indicate the presence of CAD.

Objective: To validate a previously developed 23-gene, expression-based classification test for diagnosis of obstructive CAD in nondiabetic patients.

Design: Multicenter prospective trial with blood samples obtained before coronary angiography. (ClinicalTrials.gov registration number: NCT00500617)

Setting: 39 centers in the United States.

Patients: An independent validation cohort of 526 nondiabetic patients with a clinical indication for coronary angiography.

Measurements: Receiver-operating characteristic (ROC) analysis of classifier score measured by real-time polymerase chain reaction, additivity to clinical factors, and reclassification of patient disease likelihood versus disease status defined by quantitative coronary angiography. Obstructive CAD was defined as 50% or greater stenosis in 1 or more major coronary arteries by quantitative coronary angiography.

Results: The area under the ROC curve (AUC) was 0.70 ± 0.02 (P < 0.001); the test added to clinical variables (Diamond–Forrester method) (AUC, 0.72 with the test vs. 0.66 without; P = 0.003) and added somewhat to an expanded clinical model (AUC, 0.745 with the test vs. 0.732 without; P = 0.089). The test improved net reclassification over both the Diamond–Forrester method and the expanded clinical model (P < 0.001). At a score threshold that corresponded to a 20% likelihood of obstructive CAD (14.75), the sensitivity and specificity were 85% and 43% (yielding a negative predictive value of 83% and a positive predictive value of 46%), with 33% of patient scores below this threshold.

Limitation: Patients with chronic inflammatory disorders, elevated levels of leukocytes or cardiac protein markers, or diabetes were excluded.

Conclusion: A noninvasive whole-blood test based on gene expression and demographic characteristics may be useful for assessing obstructive CAD in nondiabetic patients without known CAD.

Primary Funding Source: CardioDx.


Grahic Jump Location
Figure 1.
Study flow diagram.

Of the 1569 patients who met the study inclusion and exclusion criteria, 226 were used for gene discovery. This diagram shows the flow for the remaining 1343 patients. ID = identification.

* 19 exclusions were for inadequate sample volume, and 5 were for patient ID mismatch (male–female mismatch between clinical data and gene expression for Y chromosome–specific transcript).

Grahic Jump Location
Grahic Jump Location
Figure 2.
ROC analysis of validation cohort performance for algorithm and clinical variables.

Comparison of the combination of Diamond–Forrester score and algorithm score with Diamond–Forrester score alone. The AUC of 0.50 is shown for reference. Information was available for 525 of the 526 patients in the validation cohort. The AUCs for the 2 ROC curves are 0.721 ± 0.023 and 0.663 ± 0.025 (P = 0.003). AUC = area under the curve; ROC = receiver-operating characteristic.

Grahic Jump Location
Grahic Jump Location
Figure 3.
Dependence of algorithm score on maximum percent stenosis in the validation cohort.

The extent of disease for each patient was quantified as maximum percentage of stenosis by QCA and grouped into 5 categories; average algorithm score for each group is shown. Scores of 10, 20, and 30 correspond to 15%, 30%, and 57% likelihood of disease, respectively. A score of 14.75, corresponding to a 20% likelihood, was used for dichotomous analyses. Appendix Figure 1, shows the complete relationship of algorithm score to likelihood of obstructive CAD. CAD = coronary artery disease; QCA = quantitative coronary angiography.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 1.
Dependence of obstructive disease likelihood in the validation set as a function of algorithm score.

Dotted lines indicate a disease likelihood of 20%, which corresponds to a score of 14.75. CAD = coronary artery disease.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 2.
Results of the net benefit analysis for the gene expression algorithm score.

The solid line is the result of the net benefit analysis for the gene expression algorithm score. A score is considered positive if the risk is above the risk threshold probability (x-axis). Reference lines reflect the net benefit of a positive result in all patients (dashed line) or a negative result in all patients (dotted line; net benefit = 0).

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Reducing Patient Radiation Exposure AND Answering the question of "Does the Patient Actually Have Heart Disease?"
Posted on October 13, 2010
Richard M. Fleming
No Affiliation
Conflict of Interest: None Declared

In response to the recent article entitled "Multicenter validation of the diagnostic accuracy of a blood-based gene expression test for assessing obstructive coronary artery disease in nondiabetic patients.

First, let me say congratulations on your efforts in further understanding the role that genes may play in Cardiovascular disease. I applaud anyone trying to reduce unnecessary and inaccurate testing of heart disease, especially if the outcome may reduce patient radiation exposure. Dr. Topol and I both appeared on 20/20 in 2004 discussing "Inflammation and Heart Disease" and I have the utmost respect for his work as a scientist.

While there are several issues involved with genetic testing, not the least of which is what insurance companies or the government would do with such information. Outstanding geneticists will gladly inform people that the discussion of Nature v. Nurture has been going on since Mendel's pea studies. I have the good fortune of having a friend by the name of Dr. Gordon Harrington, who has been instrumental in both genetic testing, childhood development and intelligence (IQ) testing. He would rightly emphasize that genes provide for a predisposition while environment determines the expression of those genes. On the medical side, I am also fortunate to have a friend by the name of Dr. Henry T. Lynch, whose work lead to the discovery of BRCA1 and BRCA2. The gene's predisposition for breast cancer is well known. Less well known is that it only provides a 25% predisposition for developing breast cancer and less of a chance for the other hormonally mediated diseases of uterine and prostate cancer. Yet, every year I hear about women who undergo prophylactic mastectomies out of fear that this gene pre-destines them to have breast cancer.

Efforts to determine if a gene "may" predispose you to heart disease is scientifically interesting; but, not determinative. When resident physicians come to me after admitting a patient to the hospital for a heart attack and then proceed to tell me the patients risk factors for heart disease indicate the patient shouldn't be having a heart attack, I still stare at them with amazement. Clearly they think these risk factors determine whether the patient can or cannot have a heart attack. Having a cholesterol of 200 mg/dl doesn't mean you have heart disease, just that you are promoting your risk of developing it. These risk factors only guide us, they never tell us if the patient has heart disease. For that we need to look at the patients own heart.

So it is with genetic testing. This will never tell you if you have heart disease, only like Dr. Lynch's BRCA1 and 2, that you are predisposed to or could develop it, not that you will or have, only that you could. Given the prevalence of heart disease in this country, it's an educated bet that you will. This may NOT be where we want to utilize our limited health care funds AND it won't be "treating" anyone.

If you are looking for testing that will (1) reduce your radiation exposure, (2) tell you if you have heart disease, (3) tell you what you should be doing next and (4) reduce health care costs, I would refer you to the two papers we have recently published in peer review medical journals1,2. These studies also show that patients are receiving TOO much radiation exposure, that TOO many of them are being sent for unnecessary testing with limited or wrong results and that TOO many deaths are occurring as a result. We encourage our colleagues to read these papers1- 4 and to join the effort to continue to improve patient diagnosis and treatment while reducing unnecessary patient radiation exposure and patient risk.


1. Fleming RM, Harrington GM, Baqir R, Jay S, Sridevi Challapalli, Avery K, Green J.. The Evolution of Nuclear Cardiology takes Us Back to the Beginning to Develop Today's "New Standard of Care" for Cardiac Imaging: How Quantifying Regional Radioactive Counts at 5 and 60 Minutes Post-Stress Unmasks Hidden Ischemia. Methodist DeBakey Cardiovascular Journal (MDCVJ) 2009;5(3):42-48.

2. Fleming RM, Harrington GM, Baqir R, Jay S, Challapalli S, Avery K, Green J. Renewed Application of an Old Method Improves Detection of Coronary Ischemia. A Higher Standard of Care. Federal Practitioner 2010;27:22-31.

3. Fleming RM, Harrington GM, Baqir R. Heart Disease in Men. Chapter 3. Using Multiple Images Post-Stress to Enhance diagnostic Accuracy of Myocardial Perfusion Imaging: The Clinical Importance of Determining Washin and Washout Indicates a Parabolic Function between Coronary Perfusion (Blood Flow) and Cellular ("Uptake/Release") Function. Alice B. Todd and Margo H. Mosley Editors, Nova Publishers, 2009, pp. 75-100. (https://www.novapublishers.com/catalog/product_info.php?products _id=8409)

4. Sheikine Y, Berman DS, DiCarli MF. Technetium-99m-sestamibi redistribution after exercise stress test identified by a novel cardiac gamma camera: Two case reports. Clin Cardiol 2010;33:E39-E45.

Conflict of Interest:

None declared

Submit a Comment/Letter

Summary for Patients

Using a Gene Test to Better Identify Heart Disease in Patients With Chest Pain

The summary below is from the full report titled “Multicenter Validation of the Diagnostic Accuracy of a Blood-Based Gene Expression Test for Assessing Obstructive Coronary Artery Disease in Nondiabetic Patients.” It is in the 5 October 2010 issue of Annals of Internal Medicine (volume 153, pages 425-434). The authors are S. Rosenberg, M.R. Elashoff, P. Beineke, S.E. Daniels, J.A. Wingrove, W.G. Tingley, P.T. Sager, A.J. Sehnert, M. Yau, W.E. Kraus, L.K. Newby, R.S. Schwartz, S. Voros, S.G. Ellis, N. Tahirkheli, R. Waksman, J. McPherson, A. Lansky, M.E. Winn, N.J. Schork, and E.J. Topol for the PREDICT (Personalized Risk Evaluation and Diagnosis in the Coronary Tree) Investigators.


Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.