0

The full content of Annals is available to subscribers

Subscribe/Learn More  >
Medicine and Public Policy |

Individualized Guidelines: The Potential for Increasing Quality and Reducing Costs

David M. Eddy, MD, PhD; Joshua Adler, MHA; Bradley Patterson, MA; Don Lucas, PhD; Kurt A. Smith, PhD; and Macdonald Morris, PhD
[+] Article and Author Information

From Archimedes, San Francisco, California.


Disclaimer: The ARIC Study is conducted and supported by the National Heart, Lung, and Blood Institute in collaboration with the ARIC study investigators. This article was prepared using a limited-access data set and does not necessarily reflect the opinions or views of the ARIC Study investigators or the National Heart, Lung, and Blood Institute.

Acknowledgment: The authors thank the primary investigator of our data set work, Peter Alperin, MD, for oversight of the data set.

Potential Conflicts of Interest: None disclosed. Forms can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-0352.

Requests for Single Reprints: David M. Eddy, MD, PhD, 201 Mission Street, 29th Floor, San Francisco, CA 94105; e-mail, author@archimedesmodel.com.

Current Author Addresses: Drs. Eddy, Smith, and Morris; Mr. Adler; and Mr. Patterson: 201 Mission Street, 29th Floor, San Francisco, CA 94105.

Dr. Lucas: 7000 East Avenue, Livermore, CA 94550.

Author Contributions: Conception and design: D.M. Eddy, J. Adler, M. Morris.

Analysis and interpretation of the data: D.M. Eddy, B. Patterson, D. Lucas, K.A. Smith, M. Morris.

Drafting of the article: D.M. Eddy, J. Adler, K.A. Smith, M. Morris.

Critical revision of the article for important intellectual content: D.M. Eddy, J. Adler, M. Morris.

Final approval of the article: D.M. Eddy, J. Adler, M. Morris.

Statistical expertise: D.M. Eddy, B. Patterson, D. Lucas, K.A. Smith, M. Morris.

Administrative, technical, or logistic support: J. Adler, M. Morris.

Collection and assembly of data: D.M. Eddy, B. Patterson, D. Lucas.


Ann Intern Med. 2011;154(9):627-634. doi:10.7326/0003-4819-154-9-201105030-00008
Text Size: A A A

Background: Current guidelines focus on a particular risk factor and specify criteria for categorizing persons into a small number of treatment groups.

Objective: To compare current guidelines with individualized guidelines (that use readily available characteristics from each person to calculate the risk reduction expected from treatment and to identify persons for treatment in ranked order of decreasing expected benefit), in the context of blood pressure management.

Design: Analysis of person-specific, longitudinal data.

Setting: The ARIC (Atherosclerosis Risk in Communities) Study.

Participants: Persons aged 45 to 64 years without preexisting cardiovascular disease who currently do not receive antihypertensive treatment.

Intervention: Treatment according to the criteria of the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7 guidelines); individualized guidelines, or treatment in decreasing order of expected benefit; and random care, or treatment of persons selected at random.

Measurements: Number of myocardial infarctions (MIs) and strokes and medical costs.

Results: Compared with treating people according to random care, individualized guidelines could prevent the same number of MIs and strokes as JNC 7 guidelines at savings that are 67% greater than using JNC 7 guidelines, or it could prevent 43% more MIs and strokes for the same cost as treatment according to JNC 7 guidelines. The superiority of individualized guidelines was not sensitive to a wide range of assumptions about costs, treatment effectiveness, level of risk for cardiovascular disease in the population, or effects on workflow. The degree of superiority was sensitive to the accuracy of the method used to rank patients and to its span (the proportion of the population for whom all of the outcomes of interest can be calculated).

Limitations: Specific results apply to the effects of blood pressure management on MI and stroke in the ARIC Study population. The methods for calculating individual benefits require quantitative evidence about the relationships among risk factors, long-term outcomes, and treatment effects.

Conclusion: Use of individualized guidelines can help to increase the quality and reduce the cost of care.

Primary Funding Source: None.

Figures

Grahic Jump Location
Figure 1.
CVD events prevented and costs for JNC 7 guidelines, random care, and individualized guidelines in the ARIC Study population.

The solid line shows the cumulative number of CVD events prevented and cumulative net medical costs resulting from treating people in decreasing order of expected benefit, ranked by using the CV [Cardiovascular] Guidelines Calculator and methods described in the text. The number of CVD events prevented is based on actual MIs and strokes observed in the ARIC population. The square represents the CVD events prevented and net costs of applying the JNC 7 guidelines. The circle and diamond on the solid line represent the cumulative outcomes that would occur if people were treated in order of the ranked list, down to the points on the list at which either the same benefit (circle) or the same cost (diamond) as the JNC 7 guidelines is achieved. As the solid line shows, treating down the list to other points could achieve other outcomes. The dotted line shows the cumulative number of CVD events prevented and cumulative net medical costs of treating people down a list that is ordered randomly, and the triangles represent the points at which the same benefit or same cost as the JNC 7 guidelines are achieved. ARIC = Atherosclerosis Risk in Communities; CVD = cardiovascular disease; JNC 7 = Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; MI = myocardial infarction.

Grahic Jump Location
Grahic Jump Location
Figure 2.
Number of MIs and strokes prevented and costs for JNC 7 guidelines, individualized guidelines, and Framingham tables and JNC 7 guidelines in the Atherosclerosis Risk in Communities Study population.

The dashed line shows the results of using Framingham tables to rank persons who meet eligibility criteria for these tables. The dotted line represents use of the Framingham tables for eligible persons and applying JNC 7 guidelines to persons to whom the Framingham tables cannot be applied. JNC 7 = Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 1.
Calibration plot for the equation for MI.

The vertical axis shows the 5-year MI risk observed in the combined trials. The horizontal axis shows the 5-year MI risk calculated by the CV [Cardiovascular] Guidelines Calculator. Each dot represents a bin, and each bin contains an equal number of participants who had an outcome. Error bars are related to the observed MI rate (SD, 2). MI = myocardial infarction.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 2.
Calibration plot for the equation for stroke.

The vertical axis shows the 5-year stroke risk observed in the combined trials. The horizontal axis shows the 5-year stroke risk calculated by the CV [Cardiovascular] Guidelines Calculator. Each dot represents a bin, and each bin contains an equal number of participants who had an outcome. Error bars are related to the observed stroke rate (SD, 2).

Grahic Jump Location

Tables

References

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Disclosure of Potential Conflicts of Interest in the Evaluation of Computerized Clinical Decision Support Systems
Posted on May 30, 2011
Craig A. Umscheid
University of Pennsylvania
Conflict of Interest: None Declared

I read with great interest the recently published simulation study examining the effectiveness of hypertension treatment based on individualized guidelines versus a traditional guideline.(1) In the simulation, individualized guidelines powered by the authors' proprietary computer model reduced myocardial infarctions (MIs), strokes, and the cost of care in hypertensive patients when compared to the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7).(1) The reductions in MI and stroke resulting from the model were particularly impressive.(1) The effects were on par with pharmaceuticals we traditionally use to treat hypertension and cardiovascular risk. But I was surprised that the authors described their potential conflicts of interest at the end of the study as "none".(1) From a review of their company's website, the authors' computer model appears to be a product marketed to clients worldwide, including healthcare systems, industry, and foundations.(2) Much like I would expect a pharmaceutical company publishing on the effectiveness of a drug it manufactures to disclose a potential conflict of interest, I would expect a company publishing on the effectiveness of a decision support tool it's developed to be held to the same standard. It's unclear if this was just an isolated example of a simple oversight, or if this represents an issue that is more pervasive in published literature examining the effects of computer programs on healthcare, where potential conflicts may not be as easily recognized or identified as traditional pharma conflicts. In this new era of comparative effectiveness research, where drugs are compared alongside other treatment modalities including surgical procedures, medical devices, behavioral interventions, and computerized clinical decision support systems, it's important that we as authors, reviewers and editors be mindful about disclosing such potential conflicts.(3) In addition, with increasing investments in health information technology fueled by the HITECH Act and "meaningful use" (4), we're likely to see even more studies examining the impact of computerized decision support programs by those who have created them, ranging from simple evidence-based information resources to complex differential diagnosis generators. Having potential conflicts disclosed alongside a published study can help readers fully appraise the studies validity, and apply the results to patient care. As journals consider methods to standardize the reporting of potential conflicts by authors, particular attention should be paid to disclosures around the study of less traditional interventions such as computerized decision support systems.(5)

References:

1. Eddy DM, Adler J, Patterson B, Lucas D, Smith KA, Morris M. Individualized Guidelines: The Potential for Increasing Quality and Reducing Costs. Ann Intern Med. 2011;154(9):627-634.

2. Archimedes Incorporated: Healthcare Modeling. Available online at: http://archimedesmodel.com/. Accessed May 28, 2011.

3. Volpp KG, Das A. Comparative effectiveness--thinking beyond medication A versus medication B. N Engl J Med. 2009;361(4):331-3.

4. Blumenthal D. Launching HITECH. N Engl J Med. 2010;362(5):382-5.

5. Drazen JM, Van Der Weyden MB, Sahni P, Rosenberg J, Marusic A, Laine C, et al. Uniform Format for Disclosure of Competing Interests in ICMJE Journals. Ann Intern Med. 2010; 152(2):125-126.

Conflict of Interest:

The author collaborates with multiple organizations on the development of traditional clinical practice guidelines.

Individualization Requires Accurate Risk Estimates in Clinical Settings
Posted on May 30, 2011
Ravi Varadhan
Division of Geriatric Medicine & Gerontology, Johns Hopkins University
Conflict of Interest: None Declared

Identifying individuals who will derive more benefit than harm from the treatment is the primary goal of evidence-based medicine. Medicare spends a fortune each year on procedures that have highly questionable benefit for the individuals who received them.(1) One-fifth of all implantable cardiac defibrillators are placed in patients who, according to clinical guidelines, will not clearly benefit from them.(2) Therefore, we welcome Eddy et al.'s proposal to guide treatment on the basis of the individual's underlying risk of the primary outcome.(3) This proposal is sensible and has face validity. However, there is a fundamental problem. How accurately can we determine an individual's underlying risk for the primary outcome?

Risk prediction models tend to generalize poorly. A main reason for this is that the risk of an outcome is typically governed by numerous risk factors, each one of which only contributes a small amount to the overall risk. Typically, only a small proportion of the overall variation in survival is explainable even with the inclusion of several risk factors. Furthermore, the risk models are sensitive to the joint distribution of risk factors in the sample in which the models are developed. Hence, they tend not to predict well when applied to a population with a different multivariate distribution of risk factors. If we can identify a few blockbuster risk factors, we can hope to develop generalizable prediction models, but this is seldom the case in primary prevention.

Eddy et al. seem to have sidestepped this problem by using the ARIC sample to both estimate the risk of MI and stroke and evaluate the impact of allocating treatment on the basis of the estimated risk. This strategy does not test the generalizability of their risk calculator. It is then no surprise that their risk-based treatment allocation strategy convincingly beats the externally derived guideline-based strategy. By not using an external risk calculator, which is what a clinician would have access to in routine practice, Eddy et al. portray an overly optimistic picture of the risk-based strategy. Hence, a more relevant test of their strategy is to make the treatment allocation on the basis of an external risk calculator, and then compare it to guideline-based allocation.

In conclusion, while we welcome Eddy et.al's proposal to guide treatment allocation on the basis of underlying risk for the primary outcome, our enthusiasm is tempered by the fundamental challenge of obtaining accurate risk estimates in clinical settings.

References:

1. Redberg RF, Squandering Medicare's Money, The New York Times, The Opinion Pages, May 25, 2011.

2. Al-Khatib SM, et al., Non-Evidence-Based ICD Implantations in the United States, JAMA. 2011; 305(1): 43-49.

3. Eddy DM, et al., Individualized Guidelines: The Potential for Increasing Quality, Ann Intern Med. 2011; 154: 627-634.

Conflict of Interest:

None declared

Response to Letter to the Editor, 6-10-11
Posted on July 1, 2011
David M Eddy
Archimedes, Inc
Conflict of Interest: None Declared

We share the Umscheid's concerns about conflicts of interest, and took several steps to avoid them. The equations we used are actually not proprietary. As described in the paper, we created a relatively simple "CV risk calculator" specifically for this analysis, to be transparent and illustrate the concept of individualized guidelines, not to promote a particular model. We also stressed that our general conclusions do not depend on any particular model. We are developing methods for more accurately calculating the benefits individuals can expect from various treatments, and for implementing individualized guidelines in practice settings. However, none of the concepts or methods in the paper is proprietary.

We agree with Varadhan, Weiss and Boyd that the success of individualized guidelines depends on the development of externally valid risk models. However we disagree that this constitutes a "fundamental problem," implying that it is not possible to develop such models. First, creation of risk models does not require that there be a "blockbuster" risk factor. Risk models can include as many risk factors as desired and for which there is good evidence. Second, because the risk calculator we used in this analysis was purposefully kept simple in form (Cox proportional hazard) and included only a small number of factors, it underestimates what can be achieved with models that are more physiologically realistic and include more variables. Third, even this simple model validated well against not only ARIC data, but data from three other sources, none of which were used to build the model. Fourth, the external validity of our results can be further tested by using Framingham tables to rank the portion of the ARIC population to which the tables can be applied. For that sub-population they produced a "relative benefit ratio" only slightly lower than the CV risk calculator (1.38 versus 1.43, respectively). The weakness of the Framingham tables is their narrow span, not their accuracy. Sixth, no matter what risk model is used initially, when individualized guidelines are actually implemented their accuracy for any particular setting can be continuously improved by tracking actual outcomes over time and gradually tuning the model to the setting. Finally, we emphasize that in order to be superior to traditional guidelines, risk models do not have to be perfectly accurate - they only need to rank individuals more accurately than traditional guidelines, which sort people into two groups: "treat" and "don't treat".

Conflict of Interest:

None declared

Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Buy Now

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Related Point of Care
Topic Collections
PubMed Articles

Buy Now

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)