Question: What is the accuracy of a prediction rule for identifying patients with diabetes mellitus who are at high short-term risk for macro- and microvascular events, infectious disease, and metabolic complications?
Design: A cohort of patients, randomly split into derivation and validation data sets.
Setting: Kaiser Permanente health maintenance organization (HMO) in Oakland, California, United States.
Patients: 57 722 members of the HMO who were ≥ 19 years of age, had diabetes, and were continuously enrolled in the health plan during the 2-year baseline period. The derivation data set included 28 838 patients (mean age 61 y, 53% men), and the validation data set included 28 884 patients (mean age 61 y, 52% men).
Description of prediction guide: A “best” model and 4 simpler approaches were derived: the previous events strategy (identifies patients with previous events or related outpatient diagnoses during the baseline period), the first 3 variables of the “best” model, the numerical risk score (a summed score obtained by replacing significant model coefficients with integer values: 1.0 for a significant multivariate odds ratio [OR] between 1.1 and 1.49, 2.0 for an OR between 1.50 and 1.99, and 3.0 for an OR ≥ 2, with corresponding negative numbers for significant ORs < 1.0), and ranking on the basis of average HbA1c level during baseline.
Main outcome measures: Identification of patients at high short-term risk for macro- and microvascular, infectious, and metabolic complications.
Main results: Comparisons of the test properties of the various models for predicting each type of complication are summarized in the Table.
Conclusion: Simple prediction rules were better than HbA1c levels for identifying patients with diabetes who were at high short-term risk for complications.
Test properties of 5 models for predicting complications in diabetes (validation data set)*
| Model 2 | Micro- and macrovascular | Infectious disease | Metabolic |
| Sens | Spec | +LR | −LR | Sens | Spec | +LR | −LR | Sens | Spec | +LR | −LR |
| Best models†‡ | 72% | 73% | 2.68 | 0.38 | 72% | 71% | 2.49 | 0.39 | 83% | 70% | 2.79 | 0.24 |
| Previous events | 72% | 72% | 2.57 | 0.39 | 44% | 86% | 3.10 | 0.65 | 33% | 99% | 24.6 | 0.68 |
| 3 variables‡ | 71% | 73% | 2.63 | 0.40 | 68% | 71% | 2.35 | 0.45 | 75% | 70% | 2.52 | 0.35 |
| Risk score‡ | 74% | 70% | 2.47 | 0.37 | 67% | 71% | 2.32 | 0.46 | 82% | 71% | 2.86 | 0.25 |
| HbA1c level‡ | 31% | 70% | 1.04 | 0.98 | 38% | 70% | 1.28 | 0.88 | 59% | 70% | 1.97 | 0.59 |