A Clinical Prediction Rule for Diagnosing Severe Acute Respiratory Syndrome in the Emergency Department FREE

Which characteristics distinguish patients with severe acute respiratory syndrome (SARS)?

These investigators developed a prediction rule for SARS by using data from 2649 consecutive patients seen at 2 Hong Kong triage clinics during the 2003 SARS epidemic. The following characteristics increased the likelihood of SARS: previous contact with a patient with SARS, fever, myalgia, malaise, abnormal chest radiograph, and abnormal lymphocyte and low platelet counts. Age 65 years and older or younger than 18 years, productive sputum, abdominal pain, sore throat, rhinorrhea, and high neutrophil count decreased the likelihood of SARS.

Findings probably are not applicable to isolated cases occurring during an interepidemic period.

–The Editors

The case definition of severe acute respiratory syndrome (SARS) changed numerous times since the first reports on the presenting symptoms of patients with SARS during the 2002–2003 epidemic (1–3). The definition developed by the World Health Organization (WHO) (4) throughout the main period of spread in 2003 specifies that patients with a history of high fever (body temperature > 38 °C), cough or breathing difficulty, and 1 or more exposures (defined as being in close contact with patients with SARS or residing in or traveling through a region affected by SARS) during the 10 days before onset of symptoms would qualify as suspect cases. A probable case of SARS is defined as meeting 2 additional criteria: 1) radiographic evidence of infiltrates consistent with pneumonia or respiratory distress syndrome on chest radiographs, or 2) laboratory-confirmed results that are positive for SARS coronavirus. However, Rainer and colleagues (5) previously concluded that these WHO guidelines for diagnosing suspected cases of SARS may not be sufficiently sensitive for assessing patients before admission to a hospital and could result in substantial underdiagnosis of SARS.

In the context of triaging patients in the primary care community setting, such as in emergency departments or special clinics set up during an acute outbreak, the current WHO guidelines are not likely to identify all patients with SARS. This possibility has important consequences not only for affected patients but also for the broader public health because infected persons cannot be rapidly isolated and quarantined. The resurgence of SARS remains a distinct possibility in the coming winter months, given its uncertain origins and possible animal reservoirs (in the form of wild animal markets that are common in Guangdong, China) (6). Accurate, objective models of triage and diagnosis could help physicians and public health authorities assess patients' risks and improve decisions about isolation, quarantine, and treatment in hospitals.

We sought to develop a clinical prediction rule for diagnosis that would accurately identify patients with SARS in the emergency department setting during an outbreak and to validate the predictive accuracy of this rule.

We analyzed an integrated database, the Hospital Authority eSARS system (an electronic Web-based clinical information database developed and used during the 2003 SARS epidemic), that contained clinical and laboratory data from all patients who visited the SARS triage clinics in the emergency departments of the Prince of Wales Hospital and United Christian Hospital in Hong Kong, China. All patients seen in these 2 clinics who were subsequently hospitalized were admitted to the same hospitals; however, after recovering from the acute illness, some were transferred to Hospital Authority convalescent facilities to recuperate before final discharge home. The Hospital Authority is directly responsible for all 44 public hospitals in Hong Kong; it provides 95% of total inpatient bed-days in Hong Kong and provided for clinical care for all patients with SARS in the 2003 outbreak. Although other triage clinics were in operation during the 2003 SARS outbreak, structured, abstracted data from these clinics were unavailable for pooled analysis.

Clinical information on the history and physical examination of persons presenting to the emergency departments of the Prince of Wales and United Christian Hospitals was abstracted from chart review by trained nursing and medical officers at each hospital. The reviewers were blinded to the final diagnosis and used a standardized protocol for abstracting information. Candidate variables, collected through standardized recording forms used by triage clinic personnel, included age (<18 years, 18 to 64 years, and ≥ 65 years), sex, health care worker (yes vs. no), contact history with patients with SARS (yes vs. no), and history of travel to other SARS-affected areas (as defined by the WHO at the time) within 2 weeks of symptom onset or the time of presentation to the triage clinic (yes vs. no). Additional variables were the presence of fever (defined as either a positive self-reported history or self-measurement at home or a tympanic temperature of at least 38 °C at triage), cough, sputum, dyspnea, sore throat, rhinorrhea, chills or rigor, myalgia, anorexia, malaise, diarrhea, vomiting, abdominal pain or headache (coded as dichotomized responses), pulse rate, systolic and diastolic blood pressure, respiratory rate, and oxygen saturation on room air. Investigations consisted of chest radiography (normal, haziness, or unilateral or bilateral pneumonic consolidation as interpreted by the emergency department physician) and simple hematologic and biochemical blood tests, including hemoglobin level, leukocyte count, absolute lymphocyte count, monocyte and neutrophil counts, and platelet count. Because we did not have sufficiently complete data (<60%) on some laboratory determinations for the entire sample (alkaline phosphatase level, alanine aminotransferase level, aspartate aminotransferase level, international normalized ratio, lactate dehydrogenase level, prothrombin time, serum albumin level, and total bilirubin level), we excluded these determinations from the analysis for all patients.

According to WHO recommendations for interpreting SARS-related laboratory tests, a final diagnosis of SARS was defined (7) by 1) positive findings on reverse transcriptase polymerase chain reaction (RT-PCR) from 2 or more clinical specimens (from different sites or tested in different laboratories) obtained from patients who were alive or who had died or 2) seroconversion by enzyme-linked immunosorbent assay, immunofluorescent antibody testing, or neutralization assay against SARS coronavirus. Although most of these tests were available during the outbreak, not all patients underwent any or all of them for various reasons, including nonuniform testing protocol (especially in the earlier part of the outbreak), lack of samples because patients died and autopsy examination was not performed, and inadequate and missing specimens. Test variables such as sensitivity and specificity were unknown because no “gold standard” laboratory or clinicopathologic definitions were available for the diagnosis of this new and emerging disease, against which diagnostic test performance can be benchmarked. Immunoglobulin G antibodies against SARS coronavirus on serologic testing seemed to be the best method for confirming SARS in largely seronegative populations (8) in which the reported seropositivity rate reached 93% to 99% in Hong Kong (5, 9) and 96.2% in Toronto (10). We collected serum samples for paired serologic testing at least 21 to 28 days apart; however, anecdotal reports from longitudinal follow-up of patients with SARS in Hong Kong revealed seroconversion as long as 6 months after acute illness. Overall, Tang and colleagues (10) found that the sensitivity of a single first-generation RT-PCR assay was 54.1% in their Toronto SARS case series, assuming that all of the clinically classified patients truly had SARS. These findings were broadly similar to those reported by Peiris and colleagues (9) for the outbreak at the Amoy Gardens housing estate in Hong Kong during late March and early April 2003. In both the Hong Kong and Toronto epidemics (9–10), the peak positivity rate of RT-PCR occurred 9 to 11 days after the first appearance of symptoms; gastrointestinal specimens gave higher yields than respiratory samples. Poon and colleagues (11) later produced a second-generation RT-PCR assay capable of detecting SARS coronavirus in up to 88% of respiratory tract samples obtained within the first 3 days after illness onset in confirmed cases of SARS in the Hong Kong outbreak. This test kit had since been adopted in the latter part of the Hong Kong epidemic. We sent all specimens for SARS coronavirus testing in Hong Kong to 3 designated laboratories (Chinese University of Hong Kong, which received most of the samples from the Prince of Wales Hospital; University of Hong Kong; and the government Department of Health, which tested most of the samples from the United Christian Hospital) with rigorous quality control procedures. All 3 facilities were certified as reference laboratories by the WHO and were members of the WHO SARS Reference and Verification Laboratory Network.

From the eSARS database, we selected cohorts from the emergency departments of the 2 hospitals for the analysis. The first cohort consisted of new patients who presented to the United Christian Hospital between 10 March 2003 and 10 May 2003; the catchment area included the Amoy Gardens housing estate, a major epicenter of the SARS outbreak in Hong Kong. Most of the patients in the cohort were from Amoy Gardens (821 of 1274 [64.4%]), but patients from the surrounding Kwun Tong district, a densely populated residential and industrial area, were also included. All consecutive patients who presented to the emergency department during the recruitment period with febrile, respiratory, gastrointestinal, infectious, or otherwise constitutional or undifferentiated symptoms were diverted to the special SARS triage clinic and included in the sample. Persons who presented with conditions that were obviously not related to SARS, such as injury, trauma, myocardial infarction, or stroke, were excluded. The second cohort consisted of all consecutive new patients who presented between 12 March 2003 and 31 May 2003 to the SARS triage clinic in the emergency department of the Prince of Wales Hospital; this hospital admitted most of the initial cluster of SARS cases in Hong Kong. All hospital staff, patients, relatives of staff or patients, and the general population of the catchment district of Shatin had access to the clinic. The same inclusion and exclusion criteria described previously for the United Christian Hospital cohort were used to select patients for the Prince of Wales Hospital cohort. Figure 1 summarizes the referral pathways and clinical disposition of the 2 cohorts used to derive our prediction rule.

The prediction rule was developed in 2 steps to more closely parallel physicians' decision-making processes (12). Step 1 was designed to identify a subgroup of patients who were likely to have SARS and therefore needed additional investigation. The variables used to define this subgroup were age, sex, occupational group, contact and travel history, presenting symptoms on history taking, and physical findings. In step 2, the subgroup identified from step 1 to be at high risk for SARS was analyzed for radiologic and laboratory characteristics, in addition to significant predictors from step 1. In summary, all factors with a P value less than 0.05 from the stepwise regression in step 1 were included in the step 2 model, together with the radiologic and laboratory variables.

We identified significant (P < 0.05) predictors of a final diagnosis of SARS by bivariable chi-square tests and then entered them into a multivariable stepwise logistic regression model. We removed variables that had a P value greater than 0.05. Interaction terms were tested as candidate variables in the logistic regression model, but none of these terms entered the final models. A score-based prediction rule for a final diagnosis of SARS was developed for each step from the logistic regression equations by using a regression coefficient–based scoring method. To generate a simple integer-based point score for each predictor variable, scores were assigned by dividing β-coefficients by the absolute value of the smallest coefficient in the model and rounding up to the nearest integer. The overall risk score was calculated by adding each component together (13–14). Total score cutoffs were specified a priori for a sensitivity of 0.99 in step 1 and 0.95 for the model overall (steps 1 and 2).

After the total risk scores were computed in step 2, patients whose scores were above the threshold cutoff, and therefore at high risk for a final diagnosis of SARS, were further assigned to different risk classes by quartiles and their observed incidence of SARS was compared. The risk classification was refined to aid decision making in the allocation of different types of isolation beds on admission to the hospital (for example, general fever ward with 4 to 8 beds vs. individual negative-pressure isolation rooms). This process of allocation may be an important consideration in a large outbreak in which the surge capacity of hospitals would not be able to provide individual isolation rooms for all patients suspected of having SARS; this was a major issue in the 2003 Hong Kong epidemic.

We used cubic spline plots to examine the strength and shape of the relationships of continuous variables with the log odds of a final diagnosis of SARS (15–16). These functions were used to confirm the linearity of the associations between clinical variables and the final outcome and, thus, to inform and refine the multivariable regression models (17). Because the cubic spline plots did not reveal evidence of nonlinear relationships, all continuous variables were eventually categorized by conventional clinical groupings. For example, blood results were recoded as low, normal, or high on the basis of laboratory cutoffs; this allowed easy interpretation and application as a prediction rule in the clinical setting.

Discrimination of the models was assessed by the area under the receiver-operating characteristic (ROC) curve (18), and calibration was evaluated by using the Hosmer–Lemeshow chi-square statistic (P > 0.05 for all models) (19). Regression models were also tested for possible overfit by using linear shrinkage estimators (15–16, 20).

To examine the performance of the prediction rule, we calculated sensitivity, specificity, likelihood ratios for both positive and negative test results, and area under the ROC curve.

We validated the prediction rule internally using the bootstrap method in the original derivation data set by sampling with replacement for 1000 iterations (15–17, 20–21). Each bootstrap sample was the same size as the original derivation sample, but patients were drawn randomly with replacement from the sample. The model was refitted on each bootstrap sample following the same method adopted in the derivation process as described (including multivariable selection of predictors of a SARS diagnosis and associated 2-step risk scoring system and performance measures calculation), and then evaluated on the original derivation sample to estimate the degree to which the predictive accuracy of the model would be expected to deteriorate when applied to an independent sample of patients (17). We also computed the optimism-corrected estimates (“optimism” refers to the absolute magnitude of bias) for each performance index (sensitivity, specificity, likelihood ratios for positive and negative test results, and area under the ROC curve) (16, 22).

We performed all analyses with Stata software, version 8.0 (Stata Corp., College Station, Texas).

The Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster, which complies with the Declaration of Helsinki and its revisions, approved this study.

The funding source had no role in or influence over the design, conduct, and reporting of the study or in the decision to submit the manuscript for publication.

At the United Christian Hospital, 1274 persons presented to the emergency department; at the Prince of Wales Hospital, 1375 persons presented to the emergency department. Table 1 shows the demographic and clinical characteristics of persons on presentation to the emergency departments; the persons are stratified by SARS status. The incidence of SARS infection was 377 of 1274 (29.6% [CI, 27.1% to 32.2%]) in the United Christian Hospital cohort and 184 of 1375 (13.4% [CI, 11.6% to 15.3%]) in the Prince of Wales Hospital cohort. Thus, the rates of infection reflect those that would be expected in an acute outbreak setting and not during the interepidemic period.

In step 1 of the clinical prediction rule, age in years (18 to 64 vs. ≥ 65) and contact history were independently associated with a final diagnosis of SARS. In addition, the presence of 3 cardinal symptoms (fever, myalgia, and malaise) and the absence of sputum production, abdominal pain, sore throat, and rhinorrhea were also independently associated with a final diagnosis of SARS (Table 2). None of the vital signs achieved statistical significance in the stepwise multivariable model and were therefore excluded. Eleven percent of the cohort with a total score less than the threshold of −3 was assigned to the low-risk group and did not proceed to step 2.

In step 2, in addition to 4 of the 9 factors identified in step 1, 4 laboratory or radiographic findings (chest radiograph, lymphocyte count, neutrophil count, and platelet count) were each independently associated with a final diagnosis of SARS. Myalgia, malaise, abdominal pain, sore throat, and rhinorrhea no longer achieved statistical significance in step 2 after inclusion of the investigations. No statistical evidence of overfit, as demonstrated by linear shrinkage estimation (shrinkage factor, 0.95 [CI, 0.94 to 0.96] for step 1 and 0.83 [CI, 0.82 to 0.84] for step 2), was seen in either multivariable regression model. The point scoring system shown in Table 3 was used to quantify the magnitude of association of each of these 8 factors with SARS. A total score of 8 or greater would qualify the patient as being at high risk for SARS, with a prespecified sensitivity of 95% overall. Eight percent of those considered in step 2 were further assigned to the low-risk category.

The magnitude of the scores had good diagnostic utility. Stratification by quartile of risk score revealed a gradient in the risk for SARS. In the derivation sample, the incidence of SARS was 4.4% (CI, 3.1% to 6.0%) for those assigned to the low-risk group (in steps 1 or 2), 21.0% (CI, 15.8% to 26.9%) (risk score, 8 to 12) for quartile 1 in the high-risk group, 39.5% (CI, 30.7% to 48.9%) (risk score, 13 to 15) for quartile 2, 61.2% (CI, 53.7% to 68.3%) (risk score, 16 to 18) for quartile 3, and 79.7% (CI, 72.0% to 86.1%) (risk score ≥ 19) for quartile 4. We observed similar gradations of risk in the validation cohort (Figure 2).

Table 4 presents the model performance indices. This clinical prediction rule achieved a sensitivity of 0.90 and a specificity of 0.62 (optimism-corrected estimates) in the derivation process. The discriminative ability of the model was maintained with corresponding statistics of 0.94 and 0.57 in the internal validation exercise. In the derivation cohort, the optimism-corrected area under the ROC curve was 0.85 compared with 0.88 from the internal validation procedure. Likelihood ratios for a positive result (that is, assignment to the high-risk group after step 2) were moderately strong at 2.40 and 2.23 for the derivation and internal validation processes, respectively. Likelihood ratios for a negative result (that is, assignment to the low-risk group after step 1 or step 2) were 0.10 and 0.12 for the derivation and internal validation processes, respectively.

Our findings suggest that a simple model that uses clinical data at the time of presentation to an emergency department during an acute outbreak can predict the incidence of SARS and provide a practicable diagnostic decision aid. The clinical prediction rule achieved high sensitivity and area under the ROC curve, which were maintained on internal validation by bootstrapping. This finding is important because of the high case-fatality ratio of SARS (1) and potential public health hazards associated with a misdiagnosis of the disease. In addition, the rule could rule out SARS in a substantial proportion of persons presenting to an emergency department and, thus, yielded satisfactory specificities in both derivation and validation samples. In particular, patients can be assigned to the low-risk group in step 1 on the basis of information from the initial history and physical examination alone, which permits physicians to avoid ordering laboratory tests that are often difficult to perform in a non–emergency department ambulatory setting and that may be costly. Moreover, the scores assigned to patients in the high-risk group provide a simple method to stratify their risk for SARS into 4 categories. Categorization into these categories allows, decisions to be made about the type of isolation and treatment procedures (for example, ward allocation, level of care assignment, and initial treatment regimen) before results of more definitive tests (such as RT-PCR) become available. This is especially helpful in infectious disease outbreaks because the need to screen large numbers of possible cases can overwhelm the surge capacity of the health care system (23).

On the basis of the current evidence and depending on the availability of resources and capacity of the system, we suggest that patients assigned to the low-risk group in steps 1 or 2 be discharged home (where they would be quarantined if they reported contact with a person with SARS within the previous 10 days) and followed up daily until complete resolution of their symptoms. This strategy is important because 4.4% of patients in the derivation cohort assigned to the low-risk group eventually developed SARS. Although it is impossible to verify retrospectively, anecdotal reports during the epidemic had suggested that a much higher proportion of confirmed SARS cases were initially sent home from medical facilities during the first or second presentations. Because SARS has a mean incubation period of 4 to 5 days (24) and because the disease almost always presents as a florid clinical syndrome rather than as a subclinical infection (25–26), most patients would require follow-up of only a few days to ascertain whether they have been infected. This discharge and follow-up strategy can balance the competing demands for inpatient beds and the attendant risks of cohospitalization with SARS-infected patients in an isolation ward, where community transmission through unrecognized carriers of the virus may occur. In addition, viral shedding is minimal during the asymptomatic or early symptomatic phase, and peak infectivity usually occurs during the second week of illness (9).

In contrast, persons whose risk score exceeds the threshold in step 2 of the clinical prediction rule should be admitted to an isolation ward for further assessment and possible initiation of treatment. The allocation of such patients to different types of rooms could be guided by the magnitude of the total risk score, which is predictive of the eventual incidence of SARS. For example, patients with a risk score in the highest quartile should be cared for in individual negative-pressure isolation rooms, whereas those with scores in the lowest quartile can be housed in communal isolation wards if single rooms are not available, especially in the event of a large outbreak. Such stratification by risk scores can reduce nosocomial cross-infection among admitted patients, especially when most would turn out not to have SARS. Hong Kong's experience in 2003 showed that only one fifth to one quarter of such suspect cases returned a final diagnosis of SARS. Furthermore, given the continuing uncertainty surrounding the use of ribavirin and other antiviral agents in the treatment of SARS, the risk scores may provide some guidance for the initiation of such treatment; testing this hypothesis is beyond the scope of our study.

Before recommending the adoption of this clinical prediction rule by public health authorities in their SARS management plans, we must address several potential limitations. First, the analysis was based on data from retrospective chart review, and, therefore, the accuracy and completeness of information, as documented in the medical records, would influence the validity of the results. In addition, various laboratory reference standards for diagnosing SARS were used to develop the prediction models, and this might have introduced subtle biases in ascertaining a positive diagnosis; however, such directionality is difficult to predict. There is little reason to suspect other biases except for random misclassification and incomplete data capture, which might have reduced the statistical power of the models and biased the findings toward the null.

Second, this rule was derived by using data from an acute outbreak; in this situation, the prevalence of SARS at the time of presentation was very high. Therefore, the prediction rule may not apply to isolated cases during the interepidemic period.

Third, patients designated as low risk may have important medical or psychosocial contraindications to outpatient care. For example, during the Amoy Gardens environmental point source outbreak in Hong Kong, families in a particular apartment block were evacuated from their homes and moved to a camping facility. In such cases, it may be prudent to admit patients who may nevertheless have a risk score below the admission threshold. Likewise, for patients who will probably not adhere to follow-up instructions or for those who have frail elderly household contacts, in whom the consequences of SARS are very serious (case-fatality ratio, approximately 80% for persons >75 years of age [24]), hospital admission may be considered.

Fourth, the rule was constructed from dichotomous or categorical variables to facilitate use in practice. This may oversimplify the way physicians interpret the predictor variables. Therefore, as with all clinical practice guidelines, our rule should not supersede physician judgment in equivocal or borderline cases.

Fifth, our prediction rule did not take into account the potential diagnostic utility of other investigative techniques, such as computed tomography or rapid SARS coronavirus tests; however, these techniques are probably available only at tertiary centers and may not yield immediate results at the time of consultation in primary or emergency care (11, 27–28).

Sixth, children younger than 12 years of age accounted for fewer than 8% and 1% of the derivation and validation cohorts, respectively. Because the clinical course in these children seems to differ from that in adults and elderly persons (29), more work should be done in a larger pooled pediatric sample to confirm the applicability of this prediction rule in this age group.

Finally, although our study included about one third of all SARS cases (561 of 1755 [32.0%]) in Hong Kong and other patients who turned out not to have been infected at 2 major hospital emergency departments, it lacked external validation. We only internally validated our results by bootstrapping, partly because we believe that the 2 hospitals from which we obtained our data do not provide an appropriate external validation group for one another (they are from the same health care system and had data from the same epidemic). The performance and utility of this diagnostic and prognostic index may change in different settings and depend on the extent and characteristics (for example, nosocomial vs. community) of any future outbreak. The prediction of absolute probabilities will probably depend heavily on such local factors, and the index may perform poorly for estimating them even though it may remain robust for separating categories of risk. On a more practical level, health care providers should remember the usual limitations associated with practice guidelines and must maintain a high level of clinical suspicion, especially in the case of SARS and when the isolation wards can still cope with admitting more patients. This decision tool will be most useful in a large epidemic when the health system's surge capacity has been overwhelmed by the number of patients seeking care. Ultimately, the generalizability of the findings should be confirmed by using similar databases in other SARS-affected countries and should be prospectively validated if SARS returns. In the meantime, however, we believe that our prediction rule will provide the best evidence-based guidelines in the triage and management of suspected cases of SARS in the emergency department and primary care settings.