Narrative Review: Lack of Evidence for Recommended Low-Density Lipoprotein Treatment Targets: A Solvable Problem

Rodney A. Hayward; Timothy P. Hofer; Sandeep Vijan

doi:10.7326/0003-4819-145-7-200610030-00010

Reviews |3 October 2006

Narrative Review: Lack of Evidence for Recommended Low-Density Lipoprotein Treatment Targets: A Solvable Problem Free

Rodney A. Hayward, MD; Timothy P. Hofer, MD, MSc; Sandeep Vijan, MD, MSc

Article, Author, and Disclosure Information

Abstract

Recent national recommendations have proposed that physicians should titrate lipid therapy to achieve low-density lipoprotein (LDL) cholesterol levels less than 1.81 mmol/L (<70 mg/dL) for patients at very high cardiovascular risk and less than 2.59 mmol/L (<100 mg/dL) for patients at high cardiovascular risk. To examine the clinical evidence for these recommendations, the authors sought to review all controlled trials, cohort studies, and case–control studies that examined the independent relationship between LDL cholesterol and major cardiovascular outcomes in patients with LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL).

For those with LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL), the authors found no clinical trial subgroup analyses or valid cohort or case–control analyses suggesting that the degree to which LDL cholesterol responds to a statin independently predicts the degree of cardiovascular risk reduction. Published studies had avoidable limitations, such as a reliance on ecological (aggregate) analyses, use of analyses that ignore statins' other proposed mechanisms of action, and failure to account for known confounders (especially healthy volunteer effects). Clear, compelling evidence supports near-universal empirical statin therapy in patients at high cardiovascular risk (regardless of their natural LDL cholesterol values), but current clinical evidence does not demonstrate that titrating lipid therapy to achieve proposed low LDL cholesterol levels is beneficial or safe.

Key Summary Points

No high-quality evidence could be found that suggests that titrating lipid therapy to recommended low-density lipoprotein (LDL) cholesterol targets is superior to empirically prescribing doses of statins used in clinical trials for all patients at high cardiovascular risk.

Studies addressing benefits of achieving LDL cholesterol goals have had avoidable problems, such as reliance on ecological (aggregate) analyses, ignoring statins' other proposed mechanisms of action, and not accounting for known confounders (especially healthy volunteer effects).

Much more reliable evidence on currently proposed LDL cholesterol goals could be expeditiously produced by conducting cohort analyses of past statin trials that control for statin dose and pill adherence.

Dichotomous comparisons (such as comparing those who reach goal vs. those who do not) can mistakenly suggest that not achieving the treatment goal results in moderate risk when in fact almost all of the risk is caused by large deviations from the ideal goal.

Proposals for treatment goals should also consider the risks, patient burden, and societal costs of the treatments that may be needed to reach those goals.

In 2004, a National Cholesterol Education Program (NCEP) expert panel recommended that physicians titrate lipid therapy to reach a low-density lipoprotein (LDL) cholesterol level less than 1.81 mmol/L (<70 mg/dL) in patients at very high risk for cardiovascular events (1, 2). The panel stated that consistent and compelling evidence showed a strong relationship between LDL cholesterol level and cardiovascular outcomes down to this level (1, 2). However, others have reviewed the same literature and have concluded that there is no valid evidence from clinical trials (see Glossary) supporting this conclusion (3, 4). Since the early 1900s, we have known that familial hyperlipidemia syndromes result in premature cardiovascular disease, and in the United States and northern Europe, cohort studies have usually found that LDL cholesterol is a major independent cardiovascular risk factor at levels above 3.75 mmol/L (>145 mg/dL) (5, 6). However, these studies had limited ability to assess whether this relationship continued at lower LDL cholesterol levels, and some suggested that this association was less marked as LDL cholesterol level approached 3.36 mmol/L (130 mg/dL), especially when high-density lipoprotein cholesterol levels were normal (7, 8). Furthermore, studies in southern Europe, where LDL cholesterol levels tend to be lower in general, have often found a less strong association than those conducted in northern Europe, even in the moderate LDL cholesterol range (3.36 to 4.14 mmol/L [130 to 160 mg/dL]) (7, 8). In addition, studies in Asia and in elderly persons have often found no decrease or even an increase in cardiovascular risk when LDL cholesterol level drops below 3.36 mmol/L (130 mg/dL) (9). These results raised questions about whether the strong association found at higher levels of LDL cholesterol could be extended to lower LDL cholesterol levels; they also raised concerns that total LDL cholesterol is an unreliable marker of benefit and may be confounded by dietary factors or LDL subparticles that are the true causal factors (7–11).

These concerns seemed to be allayed when multiple clinical trials showed that statin therapy dramatically decreased cardiovascular events in almost all groups at high risk and that this benefit extended to those with pretreatment LDL cholesterol levels of 2.33 to 2.59 mmol/L (90 to 100 mg/dL) (1, 2, 12–17). Several recent trials have also shown greater benefits for high-dose statin therapy compared with low to moderate doses for those with acute coronary syndromes (14, 17) and known coronary artery disease (15, 16) (although the results in the IDEAL [Incremental Decrease in Endpoints Through Aggressive Lipid Lowering] study [16], in which participants had stable coronary artery disease, did not reach statistical significance). However, these studies generally used fixed doses of statins (placebo vs. statin or low-dose vs. high-dose statin) and therefore cannot directly shed light on whether clinicians should prescribe the doses used in the studies or titrate lipid therapy to achieve recommended LDL cholesterol goals.

This is particularly relevant because statins do much more than decrease LDL cholesterol levels. Although strong mechanistic evidence supports the LDL hypothesis, strong basic science evidence (18) also suggests that the effects of statins on inflammation, thrombosis, and oxidation are plausible mechanisms for mediating the benefits of statin therapy (often referred to as “pleiotropic effects”) (Appendix Table 1). Indeed, some statin trials seem to run counter to the LDL hypothesis. For example, trials have found that statins substantially reduce the risk for stroke, which is more consistent with their hypothesized antithrombotic effects than with their LDL-lowering effects (high LDL levels are not a major independent risk factor for stroke) (19). In addition, a recent large statin trial conducted in patients receiving dialysis found no substantial benefit despite reductions of 42% in LDL cholesterol levels (20), suggesting that even dramatic reductions are not always associated with clinically significant lowering of cardiovascular risk.

Appendix Table 1. Known Lipid-Independent Effects of Statins*

For clinicians and patients, this issue is much more than an academic debate. Compared with empirically treating patients at high cardiovascular risk with statin doses similar to those used in clinical trials, titrating lipid therapy to recommended LDL cholesterol goals entails considerably greater clinical complexity, frequent use of multidrug therapy, and greater societal and patient out-of-pocket costs; these, in turn, can result in increased patient burden and lower adherence to all treatments (21–23). Fewer than half of those receiving high doses of statins in clinical trials have achieved LDL cholesterol levels less than 1.81 mmol/L (<70 mg/dL) (15–17), and it is unclear whether achieving this goal is truly feasible even if multidrug therapy is used. Most important, if reducing total LDL cholesterol to very low levels is not truly the dominant beneficial mechanism of statin therapy, pursuit of such values using multidrug therapy could result in net harm to patients (22). This concern may be heightened by recent clinical trials suggesting that some treatments that “improve” lipid profiles, such as hormone replacement therapy and muraglitazar, actually increase cardiovascular risk (24–26).

In this paper, we examine the clinical evidence for and against recommended treatment goals for LDL cholesterol levels and outline an approach by which the benefits of these and other proposed treatment goals may be better assessed in the future.

Methods

Implicit in recommendations to pursue a specific treatment goal (such as LDL cholesterol level <1.81 mmol/L [<70 mg/dL]) is the assumption that reaching the goal is a strong predictor of the degree of patient benefit independent of all known confounders, including the treatment or treatments. Otherwise, we would simply give people the treatments used in studies. Therefore, we sought to identify studies that examined whether reaching low LDL cholesterol targets or a more substantial percentage reduction of LDL cholesterol (the 2 goals put forth in the NCEP guidelines) is a strong independent predictor of cardiovascular risk reduction.

We sought to examine all controlled trials, cohort studies (see Glossary), and case–control studies that examined the independent association between LDL cholesterol levels and major cardiovascular outcomes in patients with LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL). We began by reviewing all of the literature cited in the 2004 NCEP expert panel report (1) that proposed the new treatment goal of less than 1.81 mmol/L (<70 mg/dL). The NCEP expert report does not note whether a formal MEDLINE review was conducted, so we also reviewed all citations from the American College of Physicians lipid guidelines (3, 4), a recent meta-analysis (2), and the Cochrane database (27). In addition, we conducted an updated MEDLINE review (1 April 2004 to 20 May 2006) using the following search: ((low density lipoprotein cholesterol OR ldl cholesterol) AND (cohort study OR case–control study OR case control study OR randomized, controlled study OR clinical trial) [limited to human and adults]. All abstracts of the 1214 articles produced in this search were screened by trained masters-level research associates, and all identified controlled trials, cohort studies, and case–control studies that reported any morbidity or mortality outcomes underwent full independent review by 2 of the authors. Finally, we contacted members of the NCEP expert panel, prominent cardiovascular clinical trialists, and experts in lipid therapy and asked whether they could identify any additional studies that met our inclusion criteria ( Appendix).

We began our review by examining the literature and arguments cited by the NCEP in support of its recommendation (1). Next, we reviewed all identified experimental evidence and all valid observational evidence (controlled longitudinal studies [cohort and case–control studies] that used a multivariable regression technique to control for all known major confounders) that met our inclusion criteria (that is, assessed the independent association between LDL cholesterol level and major cardiovascular outcomes in patients with LDL cholesterol levels <3.36 mmol/L [<130 mg/dL]). Criteria for quality ratings of eligible articles were planned by protocol but were rendered unnecessary because no studies met our minimum inclusion criteria. Therefore, we examined methodologic limitations in the studies found in our literature review. Most major problems found in the experimental literature could be classified as not considering the alternative hypotheses and mistaking cohort results for true experimental results. Most major methodologic limitations in the observational literature could be classified as not demonstrating whether the LDL cholesterol association was independent of treatment exposure (that is, how much statin patients were taking), ignoring healthy volunteer effects, using ecological (aggregate) comparisons rather than true cohort (individual) analyses, and relying on dichotomies instead of examining more continuous associations.

Results

Basis of the NCEP Target

The experts from the NCEP stated the following:

Recent clinical trials nonetheless have documented … that for every 1% reduction in LDL-C [low-density lipoprotein cholesterol] levels, relative risk for major CHD [coronary heart disease] events is reduced by approximately 1%. HPS [Heart Protection Study] data suggest that this relationship holds for LDL-C levels even below 100 mg/dL [2.59 mmol/L].

The log-linear LDL hypothesis referred to by the NCEP experts suggests that independent of all other risk factors, the log of the relative risk for cardiovascular events is linearly associated with LDL cholesterol level (1). Figure 1 shows predictions of the log-linear hypothesis for 2 patients with diabetes. Although the relative risk reduction is constant in a log-linear association, the absolute risk reduction has diminishing returns (just as when a piece of paper is serially torn in half and half is thrown away, the halves that are thrown away get smaller and smaller). Sometimes these diminishing returns predict that very few absolute benefits will accrue from pursuing very low treatment goals (such as in low- to moderate-risk patients). However, Figure 1 also demonstrates how patients at high cardiovascular risk could obtain clinically significant benefit from reaching strict LDL cholesterol goals if the log-linear relationship holds for LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL).

Figure 1.

The diminishing returns of the hypothesized log-linear relationship.

The log-linear low-density lipoprotein (LDL) hypothesis suggests that relative risk reduction is constant but that there are diminishing absolute benefits. For example, reducing LDL cholesterol level by 0.78 mmol/L (30 mg/dL) is associated with a 24% relative risk reduction in all instances, but the amount of absolute benefit is much greater when LDL cholesterol level is reduced from 5.69 mmol/L (220 mg/dL) to 4.91 mmol/L (190 mg/dL) than from 2.59 mmol/L (100 mg/dL) to 1.81 mmol/L (70 mg/dL). Absolute benefit is also greater when higher-risk patients (top) compared with lower-risk patients (bottom) are being treated. The top panel predicts reduction in cardiovascular (CV) risk for a 65-year-old white woman with type 2 diabetes mellitus and systolic blood pressure of 145 mm Hg, high-density lipoprotein cholesterol level of 0.54 mmol/L (21 mg/dL), triglyceride level of 3.39 mmol/L (300 mg/dL), and hemoglobin A_1c level of 7%. The bottom panel predicts reduction in CV risk for a 60-year-old white woman with type 2 diabetes mellitus and systolic blood pressure of 125 mm Hg, high-density lipoprotein cholesterol level of 1.42 mmol/L (55 mg/dL), triglyceride level of 1.13 mmol/L (100 mg/dL), and hemoglobin A_1c level of 7%. To convert LDL cholesterol values to mmol/L, multiply by 0.02586.

Experimental Evidence for the LDL Hypothesis When LDL Cholesterol Levels Are Less than 3.36 mmol/L (<130 mg/dL)

Almost all published clinical trials examined fixed doses of statins (placebo vs. statin or low-dose vs. high-dose statins), and no published trial examined titrating lipid therapy to the proposed LDL cholesterol goals. Therefore, the main results of these trials could not be used to support or refute the benefits of titrating lipid therapy to try to achieve these LDL cholesterol targets. We found only 1 valid experimental subgroup analysis of this question, which does not support the log-linear LDL hypothesis (13). In the HPS, all participants received 40 mg of simvastatin (the study drug) before randomization and investigators measured each participant's biological response to statin therapy. This allowed a true experimental subgroup analysis. By giving all study participants a brief trial of 40 mg of simvastatin before randomization, investigators were able to match control and intervention participants according to their response to statin therapy. Contrary to the LDL log-linear hypothesis (which would suggest that those who have a larger LDL cholesterol response from a given statin dose would receive greater benefit), those with the worst prerandomization LDL response (<38% reduction in LDL cholesterol level) received the same benefit as those with the best LDL response (>48% reduction in LDL cholesterol level) (Figure 2). Therefore, we could find no experimental evidence suggesting that the degree of LDL cholesterol reduction is an independent predictor of cardiovascular risk if LDL cholesterol level is less than 3.36 mmol/L (<130 mg/dL) (13).

Figure 2.

Results for the Heart Protection Study.

No statistically or clinically significant difference was seen in relative benefit of statin therapy by low-density lipoprotein (LDL) cholesterol level at baseline or prerandomization LDL response. To convert LDL cholesterol values to mg/dL, divide by 0.02586.

Observational Evidence for the LDL Log-Linear Hypothesis When LDL Cholesterol Levels Are Less than 3.36 mmol/L (<130 mg/dL)

All potentially eligible observational studies (see Glossary) were cohort studies that used data originally collected as part of a statin trial. A cohort study is a type of longitudinal observational study in which risk factors are collected at baseline for a group of people (the cohort) and the researchers then use multivariable analysis to assess which risk factors are strong independent predictors of future outcomes. Cohort studies can be a strong source of evidence for independent associations, but only if the investigators are able to account for all likely confounders (28–30). Because none of the cohort analyses of statin trials adequately controlled for known major confounders, it is impossible to draw sound conclusions on the basis of their results.

Problems with the Experimental Evidence

Not Considering Alternative Hypotheses When Interpreting Experiments

What was the “clinical trial” evidence to which the NCEP report referred? The NCEP report emphasized that clinical trials have found that the degree of relative benefit of statin therapy in high-risk patients is largely independent of baseline LDL cholesterol level. This is consistent with a log-linear association because the log-linear hypothesis predicts that reducing LDL cholesterol levels by 0.78 mmol/L (30 mg/dL) will produce the same relative risk reduction in someone with an LDL cholesterol level of 5.69 mmol/L (220 mg/dL) and someone with an LDL cholesterol level of 2.59 mmol/L (100 mg/dL); only the absolute benefits will differ (Figure 1) (1, 13). However, this argument ignores a basic tenet of the scientific method: For evidence to advance a particular hypothesis, it must be more consistent with the proposed hypothesis than with the competing hypotheses. If the major benefits of statins are mediated through their effects on inflammation, thrombosis, and oxidation (Appendix Table 1) (18), we would also expect the relative benefits of statin therapy to be independent of baseline LDL cholesterol level. Therefore, this finding does not advance the LDL log-linear hypothesis over other possibilities and does not help us determine whether percentage LDL cholesterol reduction or absolute LDL cholesterol level is a valid indicator of the appropriate number and dosage of lipid-lowering therapies for high-risk patients.

Mistaking Cohort Analyses for True Experimental Results

Although the beauty of the true experiment (see Glossary) is that random assignment of large numbers of patients usually protects study results from both known and unknown confounders without a need for statistical adjustment, this protection applies only to the specific intervention tested (28–30). In the case of statin trials, the clinical intervention randomized is statin therapy, not LDL cholesterol levels. Although many researchers have noted that the degree of LDL cholesterol reduction in these trials is associated with the degree of cardiovascular risk reduction in clinical trials (2), this finding is based on 2 postrandomization findings and is therefore observational, not experimental. For this reason, potential confounders must be taken into account.

Problems in Cohort Analyses That Used Clinical Trial Data

Performing cohort analyses of clinical trials can be an excellent way to assess the merits of specific treatment goals (30). Such analyses are often much better able than experiments to directly assess a treatment's mechanism of action and the nature of continuous effects. However, observational analyses using clinical trial data have the same risks for confounding as those in any other observational cohort analysis, even though the data originate from an experiment. In fact, 2 notable major sources of confounding are particularly problematic in such studies, and neither was considered in any of the cohort studies found in our review.

Cohort Analyses Using Clinical Trial Data Must Control for Exposure to the Treatment

Anything that an effective treatment causes (including its side effects) can appear to improve outcomes if researchers do not control for treatment exposure (in this case, treatment exposure is a combination of statin dose and pill adherence). For example, because statin therapy increases myalgia, an uncontrolled cohort analysis of a statin trial will find that patients with myalgia have fewer cardiovascular events than those without myalgia. However, this finding would merely be due to confounding because patients with myalgia are more likely to be taking a statin.

Because clinical trials examining titrating treatments to achieve LDL cholesterol goals are in progress and because such studies are sometimes misinterpreted as demonstrating the importance of reaching a treatment target, we will examine a hypothetical dose-titration trial in more detail (Figure 3). This hypothetical trial has 3 treatment arms: 1) placebo; 2) megastatin, 10 mg; and 3) titrating megastatin (maximum dose, 80 mg) to achieve an LDL cholesterol target of less than 1.81 mmol/L (<70 mg/dL). Figure 3 shows the true results: 1) Megastatin decreases LDL cholesterol level in a dose-dependent manner (the higher the megastatin dose, the greater the reduction); 2) megastatin lowers cardiovascular risk (40 mg is better than 10 mg, but 80 mg is no better than 40 mg); and 3) although megastatin reduces both LDL cholesterol levels and cardiovascular risk, none of the benefits of megastatin are related to its effect on LDL cholesterol.

Figure 3.

A hypothetical example of the limitations of dose-titration studies and the importance of controlling for treatment exposure when examining the benefits of reaching treatment targets.

*In this hypothetical example, megastatin reduces both cardiovascular (CV) risk and levels of low-density lipoprotein (LDL) cholesterol, but reduction of the latter is not related to megastatin's benefits. The maximum relative risk reduction (RRR) for CV events is achieved with 40 mg of megastatin, and maximum reduction in LDL cholesterol levels is achieved with 80 mg of megastatin. All results can be calculated from the assumptions in section A or can be obtained from the first author by request. † The adjusted observational (cohort) analysis controls for dose of megastatin and finds that reaching the LDL cholesterol goal is not an independent predictor of degree of CV risk reduction. To convert LDL cholesterol values to mmol/L, multiply by 0.02586.

Figure 3 shows how an unsophisticated interpretation of this dose-titration experiment could lead one to mistakenly conclude that achieving an LDL cholesterol goal less than 1.81 mmol/L (<70 mg/dL) is beneficial. When compared with placebo, the “titrate to goal” study group received considerably greater benefit than the group receiving 10 mg of megastatin (relative risk reduction, 45% vs. 25%, respectively). Furthermore, a cohort analysis that does not control for megastatin dose would seem to support this conclusion, because this uncontrolled analysis will find that those who achieve an LDL cholesterol level less than 1.81 mmol/L (<70 mg/dL) benefit much more than those who do not. However, Figure 3 also demonstrates that when multivariable analysis is used to control for statin doses, LDL cholesterol level is not independently associated with cardiovascular risk reduction. In addition, such an analysis indicates that the optimal clinical strategy is to ignore LDL cholesterol levels completely and simply give all patients 40 mg of megastatin.

Although none of the cohort studies found in our literature review controlled for treatment exposure, some controlled for arm of randomization (31, 32). This may seem adequate (or even preferable), but it is not. In an experimental analysis, researchers should focus on arm of randomization (intention-to-treat analysis). However, in a cohort analysis, where mechanism of action rather than drug efficacy is being studied, researchers need to focus on actual exposure, such as treatment received and pill adherence, to examine which of the factors that the treatment affects, such as LDL cholesterol or C-reactive protein levels, are independently related to lower risk for outcomes. This is particularly important because in statin trials, up to 15% to 20% of those randomly assigned to statins do not continue to receive treatment and up to 20% to 30% of those randomly assigned to placebo cross over to statin therapy (13).

In summary, unless a cohort study adequately controls for degree of exposure to treatment, it should not be considered evidence of the importance of achieving a treatment goal. Even in a dose-titration trial, a properly conducted cohort analysis may be necessary to examine whether the benefits achieved in the study are more strongly associated with achieving the treatment goal or whether achieving the goal is a marker for receiving more medication.

The “Healthy Volunteer” Effect Can Severely Bias Studies Evaluating Treatment Targets

Even after controlling for a host of confounders, rigorous cohort studies consistently found that postmenopausal hormone replacement therapy (HRT) was an independent predictor of better cardiovascular outcomes (33). When clinical trials demonstrated that HRT actually increases cardiovascular risk (25, 26), much of the medical world was shocked, but it should not have been. It has long been known that improved adherence to treatment (even to a placebo treatment) can be a strong independent predictor of better cardiovascular outcomes (34–36). Noting this finding, Barrett-Connor (37) pointed out that many women declined HRT even when it was recommended by their physicians and that many women stopped taking HRT in the first year of treatment. Therefore, those who were taking HRT over the long term were a highly selected group, and the improved cardiovascular outcomes could be due to a healthy volunteer bias (that is, those who were receiving long-term HRT were a particularly health-conscious group) (37). When consistent findings from rigorous cohort analyses have later been proved incorrect, high levels of selection, such as self-selection (healthy volunteer bias) or provider selection (referral or transfer bias), have usually been involved (38–42). There are many similar examples in the epidemiologic literature, such as better outcomes for patients with higher intake of β-carotene (clinical trials found that supplements are harmful) or better outcomes for patients with higher vitamin E serum levels (clinical trials have found no benefit of supplements) (38–41).

Counterintuitively, the healthy volunteer bias can be particularly problematic when we use clinical trial data to conduct cohort analyses (30). For example, in a randomized trial of statin therapy, achieving an LDL cholesterol level less than 1.81 mmol/L (<70 mg/dL) is influenced in part by unbiased assignment to arm of randomization but will largely depend on tolerance to statin therapy (influenced by hardiness, comorbid conditions, and a healthy volunteer effect), level of adherence to the treatment (a healthy volunteer effect), and whether someone in the conventional treatment arm crosses over to statin treatment (a healthy volunteer effect). This presents an additional reason why controlling for treatment exposure, not arm of randomization, is critically important in a cohort analysis (Appendix Table 2) (31, 32). Bias due to selection is perhaps the most major and vexing source of confounding in observational analyses, and more advanced statistical methods, such as instrumental variables or propensity scores, should be used whenever possible to try to better account for selection (43–45). However, we could find no evidence that any of the published cohort studies that used data from statin trials considered drug adherence or drug intolerance.

Appendix Table 2. Key Factors That Influence whether a Participant Reaches a Treatment Target in a Clinical Trial

Ecological Comparisons Are a Very Weak Source of Evidence

Often, arguments for the log-linear LDL hypothesis have been based on ecological comparisons (1, 2, 15). An ecological study (see Glossary) involves comparing differences between groups of individuals (aggregate data), thus ignoring differences between individuals within those groups (for example, noting that clinical trials that achieved greater average change in LDL cholesterol level have also tended to achieve greater relative cardiovascular benefits) (1, 2, 15, 45, 46). The simplicity of presenting aggregate data can have great intuitive appeal, but it is actually an extremely weak source of epidemiologic evidence, having been characterized as “dangerous at best and disastrous at worst” (45, 46). Ecological comparisons have inherent risks related to confounding hidden within the aggregate data (“the ecological fallacy”) and often have so few data points that their sample sizes are insufficient to even allow attempts to control for confounders. Both of these problems are true of the ecological comparisons cited in support of titrating treatment to reach LDL cholesterol levels less than 1.81 mmol/L (<70 mg/dL) (2, 14).

Figure 4 presents a widely quoted ecological comparison based on the average LDL cholesterol values and average outcome rates of 12 groups (5 placebo groups and 7 statin-treated groups taken from 6 clinical trials) (14). On the surface, the top panel of Figure 4 seems to definitively show that achieving very low LDL cholesterol levels is a very strong predictor of the degree of cardiovascular risk. However, these 6 studies differ from each other in many ways, such as time period, country, baseline risk, crossover, and treatment adherence rates. Therefore, drawing a line through the 12 data points plotted against just one of these variables (LDL cholesterol level) can be arbitrary. For example, because cardiovascular outcomes have dramatically improved as a result of therapeutic advances, such as more aggressive use of angiotensin-converting enzyme inhibitors and angiotensin-receptor blockers, it is hard to justify comparing outcome rates in a study conducted from 1988–1994 (4S [Scandinavian Simvastatin Survival Study]) and a study conducted from 1998–2004 (TNT [Treating to New Targets] Study). When each study population is used as its own control (thus controlling for both time period and patient populations), the relationship between LDL cholesterol levels and cardiovascular events changes substantially, becoming less strong and less uniform (Figure 4, bottom). However, with only 12 data points, it is still not possible to control for other potential confounders between these studies, such as differences in treatment exposure (due to drug intolerance and crossover rates) and changes in other potential mechanisms of statin therapy (for example, C-reactive protein level).

Figure 4.

Summary of lipid studies.

Top. Event rate assuming a single association between low-density lipoprotein (LDL) cholesterol and outcome. Bottom. The LDL–outcome associations found within each study. 4S= Scandinavian Simvastatin Survival Study; CARE= Cholesterol and Recurrent Events Study; HPS= Heart Protection Study; LIPID= Long-Term Intervention with Pravastatin in Ischaemic Disease Study; TNT= Treating to New Targets Study. To convert LDL cholesterol values to mmol/L, multiply by 0.02586.

In contrast with these ecological comparisons, the use of individual-level data and modern multilevel modeling techniques would allow one to simultaneously analyze aggregate and individual-level effects, thus producing rigorous cohort analyses that eliminate many of the gross errors that are so common in ecological analyses (47). Finally, simply because a test is associated with better outcomes on average does not mean that the measure is sufficiently accurate to be useful in clinical practice. Patient-level analyses are needed to support patient-level recommendations (45).

Framing Treatment Goals as False Dichotomies

Finally, we did not find a single cohort analysis that tried to examine a more continuous relationship between LDL cholesterol levels and outcome rates. Although it may seem appropriate to think of treatment goals as dichotomies to be congruent with the dichotomous nature of clinical treatment decisions (intervene vs. do not intervene), such dichotomies can be misleading. The degree of absolute benefit is the difference between risks if untreated versus risks if treated and therefore always exists along a continuum that varies according to patients' individual circumstances and what treatment is being used to achieve the proposed goal (22, 30, 48). Therefore, analyses that relate benefits of a continuous risk factor (such as LDL cholesterol level or blood pressure) by measuring “treatment goal achieved” versus “treatment goal not achieved” are usually highly misleading. This is because those who do not achieve the treatment goal can represent a very heterogeneous group: those with small, moderate, and extreme deviations from the recommended goal (22). In medicine, modest deviations from “ideal” levels (for example, a hemoglobin A_1c level of 7.5% vs. a goal of <7% or a sodium level of 132 mmol/L vs. a goal of 135 to 145 mmol/L) often result in trivial risk. Marked deviations from treatment targets, however, are often associated with dramatic and often logarithmic increases in risk (22, 30, 49–52). In fact, a recent report from the Framingham Heart Study suggested that this is true for a variety of cardiovascular risk factors, including cholesterol levels (49). If the only comparison made is between those who reach the strict goal and all others, we can mistakenly think that not achieving the treatment goal results in moderate risk when almost all of the risk is caused by more substantial deviations from the goal.

The articles we reviewed often advocated for tight LDL cholesterol goals without discussing possible risks, patient burden, and societal costs associated with the treatments needed to reach those goals. This is particularly important because achieving moderate clinical control is often easy whereas achieving the ideal goal often requires substantial costs and patient burden, such as polypharmacy. Many treatments also carry at least some risk for harm. Therefore, failure to recognize this phenomenon can result in promoting unsafe treatment recommendations for those with small to moderate deviations from the proposed treatment goal (22).

Discussion

In this review, we found no high-quality clinical evidence to support currently proposed treatment goals for LDL cholesterol. However, we conclude that there are no intrinsic barriers to producing such evidence. For example, a large clinical trial like the HPS provides excellent statistical power for a cohort analysis to assess the LDL hypothesis at low LDL cholesterol levels. This cohort analysis would control for pre-event values of known cardiovascular risk factors, treatment status (placebo vs. statin, assessing interactions with deviations from arm of randomization), and pill adherence (Figure 5). To measure adherence, pill counts or pharmacy data are preferred; however, at a minimum, collecting information by patient self-report is always feasible and can be done inexpensively and easily (53).

Figure 5.

Diagram of a cohort study assessing whether lipid lowering is an independent predictor for the degree of benefit derived from statin therapy.

Experiments (top) generally assess interventions but can rarely directly assess their mechanisms of action. Cohort analyses (bottom) can provide evidence for or against proposed mechanisms of action by examining whether a marker for the proposed mechanism of action (i.e., low-density lipoprotein [LDL] cholesterol level) is an independent predictor of lower risk after controlling for treatment exposure, adherence, and other known risk factors for the outcome being studied. CV= cardiovascular.

Although conducting cohort analyses as discussed here would produce much stronger evidence regarding the hypothesized LDL log-linear relation at LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL), such evidence will still have important limitations. Any observational analysis can be incorrect because of unknown confounding, even if it controls for treatment exposure and adherence. As we and others have pointed out previously, however, clinical trials have their own limitations for adequately informing clinical practice, and never considering observational evidence is not a viable option (29, 30, 54). For example, if researchers refuse to consider high-quality observational evidence, some outlandish conclusions could result (54). No clinical trials support the benefits of cervical cancer screening, show that aspirin therapy causes Reyes syndrome in children, or prove that improved hemoglobin A_1c levels reduce long-term serious morbidity in type 2 diabetes. However, strong, persuasive observational evidence supports each of these conclusions. What we must not do is fall into the trap of dismissing high-quality observational evidence when it runs counter to our hypotheses and accept poor-quality evidence (such as ecological comparisons and cohort analyses that do not control for likely confounders) when it supports our favored position.

To be clear, we think valid cohort studies have a strong likelihood of showing that LDL cholesterol levels are a good marker of statins' benefits. Our point is not that there is strong evidence against the current recommendations; it is that there is no valid clinical evidence to suggest that using treatments other than statins to pursue proposed LDL cholesterol goals is safe or effective. Even with regard to statin therapy for patients at high cardiovascular risk, a strong argument can be made that the current evidence supports ignoring LDL cholesterol altogether and titrating to high doses of statins as tolerated, especially given the potential complexities of LDL subparticles and interactions (3, 7–11, 21). Finally, even if more valid future cohort analyses demonstrate that LDL cholesterol level is a substantial independent marker for patient benefit, it may still be preferable to suggest that large clinical trials should be conducted using multidrug lipid therapy instead of assuming that multidrug therapy is safe without such large-scale clinical testing.

We conclude that there is clear and compelling evidence that most patients at high risk for cardiovascular disease should be taking at least a moderate dose of a statin if tolerated, even if their natural LDL cholesterol level is low. We could find no published high-quality clinical evidence supporting titration of lipid therapy based on proposed LDL cholesterol targets. However, the errors in previous examinations of this issue appear to be avoidable. We strongly suggest that those with access to these data conduct further analyses to provide more valid evidence on this important clinical and scientific question.

Appendix: List of Experts in Cardiovascular Disease and Research

These experts were contacted and were asked to identify any published experimental, cohort, or case–control study that examined the independent relationship between LDL cholesterol and major cardiovascular outcomes for participants with LDL cholesterol levels less than 3.36 mmol/L (<130 mg/dL): Jane Armitage, Oxford University, Oxford, United Kingdom; Elizabeth Barrett-Connor, University of California, San Diego, San Diego, California; Robert Brook, Harvard University, Cambridge, Massachusetts; Christopher Cannon, Harvard University, Cambridge, Massachusetts; Jean-Charles Fruchart, Institut Pasteur de Lille, Lille, France; Paul Durrington, University of Manchester, Manchester, United Kingdom; Kim Eagle, University of Michigan, Ann Arbor, Michigan; Heiner Greten, Universität-Krankenhaus Eppendorf, Hamburg, Germany; Scott Grundy, University of Texas Southwestern, Dallas, Texas; Steven Haffner, University of Texas, San Antonio, Texas; Donald B. Hunninghake, University of Minnesota, Minneapolis, Minnesota; John J.P. Kastelein, Academic Medical Center of Amsterdam, Amsterdam, the Netherlands; Anthony Keech, University of Sydney, Sydney, Australia; Harlan Krumholz, Yale University, New Haven, Connecticut; John C. LaRosa, Tulane University, New Orleans, Louisiana; Richard Peto, Oxford University, Oxford, United Kingdom; Bertram Pitt, University of Michigan, Ann Arbor, Michigan; Paul Ridker, Harvard University, Cambridge, Massachusetts; Melvin Rubenfire, University of Michigan, Ann Arbor, Michigan; D.L. Sprecher, Cleveland Clinic, Cleveland, Ohio; and Nanette K. Wenger, Emory University, Atlanta, Georgia.

Glossary

Clinical trial: A type of true experiment conducted on humans, usually used to describe studies that evaluate an intervention's impact on clinically important patient outcomes (such as morbidity or mortality). Large clinical trials are the gold standard for evaluating the effectiveness and safety of clinical interventions (grade A evidence). However, experimental analyses of clinical trials are usually unable to provide direct experimental evidence regarding the intervention's mechanism of action. Clinical trial data are often used to conduct cohort analyses, which can often examine whether evidence for or against a treatment target (such as low-density lipoprotein cholesterol) is an independent predictor of patient benefit, but such analyses must control for all potential confounders just like any other observational analysis.

Cohort study: A longitudinal, observational study in which potential risk factors are identified in a group (cohort) of study subjects at baseline and independent associations between these baseline risk factors and future outcomes are assessed by using multivariable regression techniques. Cohort analyses are often conducted using clinical trial data to evaluate study questions that cannot easily be evaluated by the true experiment, such as examining potential mechanisms or markers for the intervention's degree of benefit (or harm).

Ecological study: An observational design that can be either cross-sectional or longitudinal and examines risk factors and outcomes of groups by using aggregate data (such as noting that countries with high-fiber diets have lower rates of colon cancer, or that statin trials with greater average reductions in low-density lipoprotein cholesterol have greater average relative risk reductions). Ecological studies are sometimes useful for hypothesis generation but are generally considered an extremely poor source of evidence for hypothesis testing.

Observational study: A general term used to describe nonexperimental epidemiologic research. With rigorous control for potential confounders, some observational study designs (especially cohort and case–control studies) can produce strong grade B evidence for causal influence (but not grade A evidence). However, some observational designs (such as ecological and cross-sectional designs) are generally considered extremely weak sources of causal evidence.

True experiment: An epidemiologic study design in which subjects are randomly assigned to different treatment arms. When the sample size is sufficiently large (to make clinically important chance differences between the randomized arms very unlikely), true experiments are the gold standard for evaluating whether an intervention results in benefits or harms.