Performance of 4 Clinical Decision Rules in the Diagnostic Management of Acute Pulmonary Embolism: A Prospective Cohort Study Free

The introduction of standardized clinical decision rules (CDRs) to determine the clinical probability of pulmonary embolism (PE) has improved the diagnostic work-up of patients with suspected PE. A CDR result of “PE-unlikely” in combination with a normal d-dimer result can exclude PE in a large proportion of patients who present for evaluation (20% to 40%), without the need for additional imaging with computed tomographic pulmonary angiography (CTPA) or ventilation–perfusion (V̇/Q̇) scintigraphy, both of which involve radiation and intravenous contrast or radioisotopes. In these patients, anticoagulant drugs can be safely withheld (1–4).

Several CDRs that incorporate information from medical history and physical examination have been developed and validated. In addition to 6 objective variables, the Wells rule contains 1 subjective variable: The physician should consider the possibility of a diagnosis other than PE for the patient's symptoms (Table 1) (5). In contrast, the more recent revised Geneva score comprises 8 objective clinical variables (6). Both scores assign different weights to the variables, meaning that depending on the variable, 1, 1.5, 2, 3, 4, or 5 points need to be assigned (Table 1). Because miscalculations can occur, the scores have been simplified (Table 1) (7, 8).

Until now, the simplified Wells rule and the simplified revised Geneva score have not been validated prospectively. Also, although some of the scores have been retrospectively or prospectively compared with each other (9–12), the 4 scores have never been directly compared for their performance in excluding PE in combination with a normal d-dimer result. Therefore, we did a prospective, multicenter study on clinical accuracy to assess and directly compare the performance of 4 CDRs (Wells rule, revised Geneva score, simplified Wells rule, and simplified revised Geneva score) in excluding PE in combination with a normal d-dimer result by using a computer-based program to calculate the CDR scores.

We performed a prospective, multicenter cohort study on clinical accuracy of 4 CDRs in consecutive patients with a suspected first episode of acute PE. The study population consisted of consecutive outpatients and inpatients in whom a first acute PE was clinically suspected. Clinically suspected acute PE was defined as sudden onset of dyspnea, deterioration of existing dyspnea, or sudden onset of pleuritic chest pain. Patients were included from 7 participating academic or nonacademic hospitals in the Netherlands.

Exclusion criteria were age younger than 18 years, life expectancy less than 3 months, treatment with therapeutic-dose low-molecular-weight heparin or unfractionated heparin that was initiated 24 hours or more before eligibility assessment, treatment with vitamin K antagonists, previous diagnosis of PE, contraindication to helical computed tomography (CT) because of allergy to intravenous iodinated contrast or renal insufficiency (creatinine clearance <30 mL/min per 1.73 m² by using the Cockroft–Gault formula), pregnancy, and inability to return for follow-up. Institutional review boards of all participating hospitals approved the study protocol, and written informed consent was obtained from included patients.

Included patients had a sequential work-up involving clinical probability assessment, d-dimer testing, and CT. Treating physicians assessed the items of 4 CDRs in all patients (Table 1). In addition, a high-sensitivity quantitative d-dimer test was done (VIDAS d-dimer assay, bioMérieux, Marcy-l'Étoile, France; Tina-quant assay, Roche Diagnostica, Mannheim, Germany; STA Liatest D-Di, Diagnostica Stago, Asnières-sur-Seine, France; or Innovance d-dimer, Siemens, Marburg, Germany) for all included patients, regardless of CDR results. The type of d-dimer assay that was used depended on the local practice. Pulmonary embolism was considered unlikely if the Wells rule score was 4 points or less, the simplified Wells rule score was 1 point or less (7, 10), and the simplified revised Geneva score was 2 points or less (Table 1) (8). The revised Geneva score was formerly available only in a 3-category scheme, but it was recently made into a 2-category scheme, similar to the other scores. This was done by using a beforehand calculation of the optimal cutoff in an existing cohort of patients with suspected PE (8) for whom the revised Geneva score variables were available to calculate the score. The optimal cutoff point was determined by calculating the area under the receiver-operating characteristic curve, and the proportions of patients classified in the likely and unlikely categories were calculated (²). Patients were considered PE-unlikely if they scored 5 points or less (Table 1). A score greater than the respective cutoff indicated a classification of PE-likely for any of the 4 CDRs.

Clinical care was guided by the results of the CDRs and d-dimer testing (Figure 1). When PE was considered unlikely according to all 4 CDRs in combination with a normal d-dimer result (cutoff <500 µcg/L), PE was excluded. In all remaining patients (those considered PE-likely according to ≥1 of the CDRs or an abnormal d-dimer result), CT was indicated to confirm or exclude the diagnosis. Patients with CT indicating PE received anticoagulants; this treatment was withheld from all patients in whom the diagnosis was excluded. These latter patients were followed for 3 months. Figure 1 shows the study flow.

Standard contrast-enhanced multidetector CT was done by using a 4-slice, 16-slice, or 64-slice multidetector CT scanner with acquisition of 0.5- or 1-mm sections (depending on the weight of the patient) of the entire chest for diagnosing or excluding PE. The rotation time was 0.4 second, and the pitch factor was 1.4. The tube current was 250 to 300 mA, and the tube voltage was 100 kV. Acquisitions were done during a single breath-hold lasting 10 to 12 seconds or less, depending on the type of scanner. Eighty to 100 mL of contrast agent was injected in the antecubital vein at an injection rate of 4.0 mL/sec. Acquisition of the static pulmonary angiography scan was started after detection of automated threshold enhancement in the pulmonary trunk. A threshold difference of 100 Hounsfield units was selected for starting the acquisition. Skilled radiologists read the CT scans to determine whether PE was present or could be excluded. The radiologists were aware of an indication for CT but not whether this was based on a high CDR result, an elevated d-dimer result, or both. The diagnosis of PE was confirmed by the presence of at least 1 filling defect in the pulmonary artery tree. Management of patients with an inconclusive CT scan result was left to the attending physician and could include repeated CT, V̇/Q̇ scintigraphy, or conventional pulmonary angiography.

The treating physicians did clinical evaluations and collected data at baseline. In each participating center, a study coordinator was available for advice about the study. This coordinator also checked the completeness and correctness of the data. Demographic data and additional relevant information (for example, recent trauma or surgery, cancer, use of anticoagulant drugs, duration of time since symptom onset, and d-dimer result) were collected on a case-report form that was available in paper and digital format. The computerized design forced the physician to start the diagnostic process with a clinical evaluation of the patient and to enter all variables necessary to calculate the 4 CDRs and the d-dimer result into the computer. The computer program calculated the 4 individual CDR scores and, after combining these scores with the d-dimer test result, indicated the next recommended step in the diagnostic process according to the predefined study flow: exclusion of PE on the basis of CDR and d-dimer result, or need to perform CT (Figure 1).

Patients in whom PE was excluded were followed for 3 months, either on the basis of the CDR and d-dimer combination (for all 4 CDRs) or on the basis of a normal CT scan. All patients were instructed to return to the hospital if symptoms of venous thromboembolism (VTE) (PE or deep venous thrombosis [DVT]) or bleeding events occurred. Objective diagnostic tests (for example, CT, V̇/Q̇ scintigraphy, or compression ultrasonography) were done if VTE was suspected. One of the study coordinators interviewed patients by telephone at the end of 3 months and questioned patients about health-related events during the previous 3 months, especially for symptoms suggestive of PE or DVT, interval initiation of anticoagulant drugs, and possible hemorrhagic complications. If relevant, the patient's general practitioner was contacted for additional information. If a patient died, the cause of death was obtained from hospital records, autopsy reports, or information from the general practitioner. Deaths were classified as caused by PE if the cause of death was confirmed by an autopsy report, if an objective diagnostic test was positive for PE before death, or if the cause of death could not completely be explained by reasons other than VTE. A panel of 3 experts adjudicated all outcomes.

On the basis of a β value of 10% (power, 90%) and an α value of 0.05, we calculated that 128 positive CT results would be needed to detect a difference in sensitivity of more than 5 percentage points (55% vs. 50%) between the 2 primary CDRs (the Wells rule and the revised Geneva score). On the basis of a 20% prevalence of PE (2), a sample size of 753 participants with suspected PE was required. All additional sample-size calculations on other outcomes necessitated a smaller sample size. Because of possible withdrawal, we aimed for a total sample size of 800 patients.

In this study, the 4 CDRs were directly compared for their performance in determining whether patients had PE. This included 4 primary analyses: 1) the ability of each CDR to correctly categorize patients with suspected PE as unlikely or likely; 2) the proportion of patients in whom PE was excluded on the basis of an unlikely CDR result combined with a normal d-dimer test result at the time of the acute evaluation; 3) the safety of clinical management based on each CDR and d-dimer combination to exclude the diagnosis—true-negative results (proportion of patients safely managed without CT) and false-negative results (the VTE rate during the 3-month follow-up in patients in whom PE was considered ruled out by the initial diagnostic work-up and who did not receive anticoagulant drugs during follow-up); and 4) the distribution of patients in the probability categories according to the 4 CDRs, which was studied by using sensitivity, specificity, and receiving-operating characteristic analysis. Also, the discordant cases (patients classified as unlikely by one CDR but likely by another) were described. The reference standard in patients in whom CT was not indicated was the recurrent VTE rate during 3-month follow-up. For patients who needed CT, the reference standard was CT and 3-month follow-up.

Performance of the 4 CDRs and the combination of the CDRs and d-dimer testing was examined by using sensitivity, specificity, receiving-operating characteristic analysis, event rates, and predictive values. To assess differences among the 4 CDRs in sensitivity, specificity, and predictive values and to compare the categorization of patients into the probability groups (paired data), the McNemar test was used to compare each CDR with the other CDRs individually.

Furthermore, stratification by type of hospital (academic and nonacademic) was done to give insight into the possible type of hospital-associated differences by using a stratified Mantel–Haenszel test (CDR likely or unlikely vs. outcome of PE stratified by academic versus nonacademic hospitals). Exact 95% CIs around the observed incidences were calculated by using CI analysis (13). Descriptive variables were calculated by using SPSS software, version 16.0 (SPSS, Chicago, Illinois). Mean values and frequencies, such as the clinical characteristics of subgroups, were compared by using a t test and chi-square test, respectively. Statistical significance was set at P < 0.05.

This study was supported by unrestricted grants from the Academic Medical Center, VU University Medical Center, Rijnstate Hospital, Leiden University Medical Center, Maastricht University Medical Center, Erasmus Medical Center, and Maasstad Hospital. The boards of the respective hospitals had no specific role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for review.

From July 2008 to November 2009, a total of 1023 consecutive patients with clinically suspected PE were screened, of whom 195 (19%) were excluded because of 1 or more of the predefined exclusion criteria (Figure 2). In addition, 21 patients declined to give informed consent. The final study population of 807 participants included 644 (80%) outpatients and 163 (20%) inpatients. The baseline demographic and clinical characteristics of the 807 study participants are shown in Table 2.

Patients were managed according to the results of CDRs combined with the d-dimer test result (Figure 2). Discordant CDR results were observed in 243 patients (29%), whereas results were concordant in 564 patients. In total, PE was ruled out by a combination of an unlikely CDR result according to all 4 CDRs and a normal d-dimer result in 169 patients (21%). In 638 patients (79%), CT was indicated either because of an abnormal d-dimer result (265 patients) or because at least 1 of the CDRs indicated that PE was likely (373 patients).

d-Dimer testing was not done in 19 patients (protocol violations). This happened in 1 patient classified as PE-unlikely according to all 4 CDRs. This patient was regarded as having a positive d-dimer result, and CTPA was done (this patient was 1 of the 265 patients with an abnormal d-dimer result). In 18 other patients, the CDR results were discordant. The missing d-dimer result had no effect on the next step in the strategy because CTPA was needed on the basis of discordant CDRs.

Protocol violations for CTPA occurred in 16 patients. In 9 of these patients, CT was indicated but not done; these patients were all followed for 3 months. In 7 patients, CT was done even though it was not indicated, and it showed PE in 1 of these patients.

In total, CT confirmed the diagnosis of PE in 185 patients: 184 with at least 1 CDR and positive d-dimer result indicating PE-likely and 1 for whom CT was not indicated on the basis of study criteria but was done because of clinical judgment (patient 1 in Appendix Table 1; Figure 2). The diagnosis was excluded in 435 patients. In 164 patients, an alternative diagnosis for the symptoms was found. Computed tomography was inconclusive in 10 patients: Repeated CT excluded PE in 2 of these patients, and V̇/Q̇ scintigraphy excluded it in 2 other patients. In 1 patient, anticoagulant treatment was started on the basis of an inconclusive CT scan combined with high clinical suspicion of PE, and in another, thrombosis of the subclavian vein was found on the same scan; these patients received treatment accordingly. In the remaining 4 patients with inconclusive CT scans, the diagnosis was considered to be excluded without further testing, and as a result, these patients did not receive treatment with anticoagulant medication. A final diagnosis was established within 1 hour in most of the patients or, at maximum, within 24 hours after presentation. The overall prevalence of PE in the total study population was therefore 185 of 807 patients (23% [CI, 20% to 26%]).

In 7 of 169 patients in the all-unlikely group who had a normal d-dimer result, the protocol was violated and CT was done even though it was not indicated. In 1 of these patients, PE was diagnosed. This was regarded as a diagnostic failure in the CDR and d-dimer combination strategy (1 of 169 patients; 0.6% [CI, 0.02% to 3.3%]) (patient 1 in Appendix Table 1; Figure 2). None of the remaining 168 patients in this group received treatment with anticoagulant drugs during follow-up, and all of these patients had an uneventful follow-up. Nine patients in whom CT was indicated but not done did not receive treatment and had an uneventful follow-up. Of the 435 patients in whom PE was excluded with CT and the 8 patients who had inconclusive results and did not receive treatment, 10 (2.3%) received anticoagulant drugs during follow-up for reasons other than VTE. Seven of the 433 patients with a normal or inconclusive CT result who did not receive anticoagulants for other reasons returned with symptomatic and objectively confirmed VTE events during 3-month follow-up (patients 2 to 8 in Appendix Table 1; Figure 2). Eighteen patients died during follow-up. In 1 of these patients, DVT had already been diagnosed during follow-up; in another patient, PE was excluded as the cause of death by autopsy report; whereas in the remaining 16 patients, the cause of death was adjudicated to be unrelated to a possible VTE. Therefore, the failure rate of a normal or inconclusive CT scan in this study was 7 in 433 (1.6% [CI, 0.7% to 3.3%]). One patient (of 807 [0.1%]) was lost to follow-up. In a worst-case scenario in which this patient would have developed VTE, the failure rate after CT that excluded PE would have been 8 in 433 (1.9% [CI, 0.8% to 3.6%]). Allergy to the intravenous iodinated contrast or contrast-induced nephropathy was not recognized in the included patients during the study.

Table 3 shows how the patients were categorized by the 2 probability categories of the 4 CDRs without taking the d-dimer results into account. The proportion of patients classified as PE-unlikely was similar for the 4 CDRs. Also, the prevalence of PE in the unlikely categories was similar. Overall, the proportion of patients classified as PE-likely was largest when the simplified Wells rule was used 38% versus 28% to 32% with the other 3 CDRs (Table 3). The sensitivity and specificity of each CDR alone (without d-dimer results) ranged from 49% to 65% and from 70% to 80%, respectively (Appendix Table 2). The receiver-operating characteristic curves for the 4 CDRs were similar, and areas under the curve ranged from 0.69 to 0.73 (Appendix Figure 1).

Combined with a normal d-dimer result, the 4 CDRs excluded PE in similar proportions of patients, ranging from 22% to 24% (Table 3). The 3-month VTE failure rates did not differ among the CDR and d-dimer test result combinations and ranged from 0.5% to 0.6% (Table 3). The 95% CI was 0% to 3% for all CDRs (Table 3). When combined with the d-dimer test result, the sensitivities of the various CDRs did not differ, although there were small differences in specificity (Table 4).

Of the 434 patients with all 4 CDRs indicating PE-unlikely, 52 (12%) received a diagnosis of PE; all patients except 1 had an abnormal d-dimer result, the latter of which indicated the need for CT.

In 243 of 807 patients (29%), discordance among CDRs was observed (Figure 2). The d-dimer result was normal in 29 patients and abnormal in 196 patients, and the test was not done in 18 patients. In the latter 243 patients, CT was done, which confirmed PE in 1 patient.

The number of discordant cases between 2 scores ranged from 25 of 807 patients (3.1%) between the revised Geneva score and the simplified revised Geneva score to 199 of 807 patients (25%) between the Wells rule and the revised Geneva score (Appendix Table 3). The agreement was greatest between the original scores (Wells rule and revised Geneva score) and their simplified versions: Discordance was seen in 11% of the total cohort between the Wells rule and the simplified Wells rule and 3.1% between the revised Geneva score and simplified Geneva score. Discordance was greater than 20% between all other scores.

Despite the discordant scores, PE was not missed in any patients in the discordant group who had a normal d-dimer result (Table 5). Therefore, when combined with d-dimer results, the scores performed equally well in excluding PE.

Both inpatients and outpatients were included. The proportions of inpatients who were categorized as PE-unlikely were 37% for the simplified Wells rule, 48% for the revised Geneva score, 50% for the simplified revised Geneva score, and 57% for the Wells rule. These proportions were smaller than the proportions of outpatients categorized as PE-unlikely: 68%, 74%, 77%, and 76% for the 4 CDRs, respectively (multiple tests; all P < 0.01).

The failure rate of excluding PE on the basis of an unlikely CDR and a normal d-dimer result was similar for both inpatients and outpatients with all 4 CDRs. However, the proportion of inpatients in which PE could be excluded noninvasively was very low: Only 3 inpatients for the simplified Wells rule (3 of 163 patients [1.8%]), 4 patients for the Wells rule (2.5%), and 5 patients for the revised Geneva score and the simplified revised Geneva score (3.1%). No failures occurred in the inpatients in whom PE was excluded without the need for CTPA.

In total, 5 academic hospitals included 598 (74%) patients, whereas 2 nonacademic hospitals included 209 patients (26%). Demographic characteristics (Table 2) did not differ for patients from academic versus nonacademic hospitals, except for cancer (16% vs. 8.1%; P < 0.001) and recent surgery or immobilization (26% vs. 11%; P < 0.001). After the results for academic and nonacademic hospitals were adjusted for, probability (PE-unlikely or PE-likely) was correctly categorized more often at nonacademic sites (75% to 79% versus 66% to 71% at academic hospitals; P < 0.001 for all 4 CDRs).

Our accuracy study directly compared 4 CDRs to assess the probability of PE and showed that the CDRs are similar in 1) their ability to categorize patients as having unlikely or likely clinical probability, 2) the proportion of patients in whom CTPA was not indicated on the basis of an unlikely CDR result and a normal d-dimer result, and 3) the 3-month failure rate for VTE in patients in whom PE was excluded by CDR and d-dimer testing. Of note, although discordance in the categorization of patients as having unlikely or likely probability by the scores was present in 30% of the patients, this did not result in a difference in failure rates when the CDR results were combined with the d-dimer results.

Our results are important for and relevant to clinical practice. Despite the debate on the subjective variable in the Wells rule, in this direct comparison the Wells rule and the simplified Wells rule were equivalent in performance compared with the fully objective revised Geneva score. In addition, we were able to validate the performance of the recently introduced simplifications of the Wells rule and revised Geneva score. Both simplified scores had similar diagnostic performance compared with their original and extensively validated versions. Despite discordances between the CDR outcomes in 30% of patients, there was no difference in safety when using a management strategy based on any of the CDRs combined with d-dimer testing. This equal performance could be explained by the use of a highly sensitive d-dimer test in patients with a CDR indication of PE-unlikely.

The importance of estimation of clinical probability has been emphasized (14–18). Although d-dimer assay is sensitive in the diagnosis of PE, false-negative results are more likely when the pretest clinical probability is high (18). This prospective validation of the simplified CDRs has relevant and practical implications because they enable easier computation of the clinical probability score, which in turn could lead to better implementation of CDRs in daily clinical care.

Our findings are in line with previous studies that used these CDRs in a 2-category scheme (14). With the Wells rule, 51% to 84% of patients were categorized as PE-unlikely in previous reports, with a prevalence of PE ranging from 3.4% to 12%, compared with 72% categorized as PE-unlikely in this study and a 15% prevalence of PE. By using the simplified Wells rule, the proportion of patients classified as PE-unlikely was slightly lower in our cohort (62%) than in a previous validation study (78%), but the prevalence of PE in this PE-unlikely group was similar (13% in both studies) (10). Likewise, in an earlier retrospective study, the simplified revised Geneva score classified 65% of patients as PE-unlikely compared with 62% in the current analysis, with a 13% prevalence of PE in the previous study and a 16% prevalence of PE in our study (8). Because we believe that this is the first study to report a 2-category scheme for the revised Geneva score, we cannot compare our data on the 2-category revised Geneva score with previous findings. However, the 69% of patients with an unlikely CDR result according to the revised Geneva score and the 16% prevalence of PE in this group overlap well with the results from the other 3 decision rules.

Four highly sensitive but different d-dimer assays were used. Because CDRs were in all patients, the types of d-dimer assays were equally represented in the different CDR groups, which enabled comparison of the CDRs regardless of the d-dimer assay. The type of assay was not based on randomization but depended on the preference of the study center. Therefore, comparisons among the d-dimer assays are limited by the sample sizes.

After several retrospective or small prospective comparisons, to our knowledge this is the first large study to directly compare the most widely used CDRs (original Wells rule and original revised Geneva score) in the diagnostic management of PE. Furthermore, our study prospectively validated the performance of the recently introduced simplified Wells rule and simplified revised Geneva score. Calculation of the scores in all patients allowed direct comparison of the CDRs in a single patient population. Also, because of the computer-aided design of the study, calculation errors were minimized. Likewise, the use of a computer program to guide the physician to the next step in the diagnostic algorithm excluded the possibility that the physician's preference for a certain CDR would influence the management of a patient.

The results of our study may be applicable in a wide range of clinical settings. First, the clinical characteristics of the patients are similar to those in other population-based studies (2, 4), and the 23% prevalence of PE in this cohort is similar to that in other reports (2, 4, 6). In addition, consecutive patients were included from both academic and nonacademic medical centers.

Our study has potential limitations. First, a randomized, controlled trial of the 4 CDRs is an alternative study design, but in view of the reasonably high concordance rates, it would probably have been very inefficient. In addition, by study design, CT was done in all patients with discordant CDRs and ensured that an imaging diagnosis was available in all of those patients. The diagnostic protocol was violated in 4 patients, in whom CT was not done despite discordance of the CDRs. Three-month follow-up, however, was uneventful in these patients. Second, managment was not based on 1 CDR in combination with d-dimer testing; rather, it was based on the combination of the 4 CDRs and d-dimer testing. According to the protocol, all patients with discordant CDR results had CT. Most of these patients had elevated d-dimer levels and would have an indication for CT, even if only 1 of the CDRs was used for decision making. Only patients with discordant results and a normal d-dimer level (29 patients [3.6% of the included patients]) did not have an indication for CT according to 1 of the separate rules combined with d-dimer testing. These patients had CT because at least 1 of the other rules indicated that PE was likely. Third, use of a computerized decision-support system improves the diagnosis of PE (19). However, in daily clinical practice, these systems may not be widely available, and our results may therefore differ from a setting in which more miscalculations are possible. Finally, although both inpatients and outpatients were included in this study and no failures occurred in the patients in whom PE could be excluded, we could not validate that any of the CDR and d-dimer combinations can safely exclude the diagnosis in inpatients.

Further research may include an outcome study using 1 of the simplified CDRs in combination with d-dimer testing. Because patients with suspected recurrent PE were not included, the performance of the CDRs in this group will need additional research.

In conclusion, the Wells rule, the revised Geneva score, the simplified Wells rule, and the simplified revised Geneva score, in combination with a d-dimer test result, all performed similarly in the exclusion of acute PE. This prospective validation indicates that the simplified, more straightforward CDRs may be used in clinical practice. Which rule a physician will use should depend on local preference and acquaintance to accomplish correct use of the CDR and prevent miscalculations.

For a list of members of the Prometheus Study Group, see ².

Academic Medical Center: L.F. Beenen, P.M.M. Bossuyt, H.R. Büller, E.J. van den Dool, R.A. Douma, and P.W. Kamphuisen; VU University Medical Center: H.M.A. Hofstee and M.H.H. Kramer; Rijnstate Hospital: M.M. Hovens, K. Kaasjager, T.A.C. Nizet, and E.F. Ullmann; Leiden University Medical Center: H.C.J. Eikenboom, C. Heringhaus, M.V. Huisman, F.A. Klok, K.W. van Kralingen, L.J.M. Kroft, I.C.M. Mos, and A. de Roos; Maastricht University Medical Center: K. Boomars, H. ten Cate, and P.M.G. Erkens; Erasmus Medical Center: M.F. Durian, M.J.H.A. Kruip; Maasstad Hospital: A.A. van Houten.

The revised Geneva score was dichotomized before the start of this study. The other 3 CDRs already have a dichotomized score, and the participating hospitals were already using 2-level CDRs, so it was easier to incorporate a dichotomized score in this study design. We searched for the optimal cutoff level by using an existing cohort of patients with suspected PE (2) for whom the revised Geneva score variables were available for calculation of the score. We chose the optimal cutoff on the basis of the optimal combination between sensitivity and specificity, receiver-operating characteristic curve analysis, and frequency tables. As can be seen in Appendix Table 4 and Appendix Figure 2, a cutoff value of 5 or less revealed an area under the curve of 0.664. With this cutoff, the prevalence of PE in the unlikely category was 10% compared with 32% in the likely category, which is similar to rates of PE in other dichotomous schemes (14). Furthermore, from a practical standpoint, by using a cutoff of 5 or less, a substantial proportion (70%) of patients had a CDR outcome below this cutoff, whereas only 30% had a CDR outcome above the cutoff. Adhering to the diagnostic algorithm, 30% of patients would have to proceed directly to CTPA. In comparison, with a cutoff value of 4 or less, the CDR indicated that PE was likely in 45% of patients, meaning that they would all directly have an indication for CTPA. The area under the receiver-operating characteristic curve did not differ significantly with that of a cutoff of 5 or less (0.668 vs. 0.664) (Appendix Figure 2).