The full content of Annals is available to subscribers

Subscribe/Learn More  >
Articles |

A Computerized Handheld Decision-Support System to Improve Pulmonary Embolism Diagnosis: A Randomized Trial

Pierre-Marie Roy, MD, PhD; Pierre Durieux, MD; Florence Gillaizeau, MS; Catherine Legall, MD; Aurore Armand-Perroux, MD; Ludovic Martino, MD; Mohamed Hachelaf, MD; Alain-Eric Dubart, MD; Jeannot Schmidt, MD, PhD; Mirko Cristiano, MD; Jean-Marie Chretien, MS; Arnaud Perrier, MD; and Guy Meyer, MD
[+] Article, Author, and Disclosure Information

For a list of the SPEED investigators, see the Appendix.

From Centre Hospitalier Universitaire d'Angers and Université d'Angers, Angers, France; Hôpital Européen Georges Pompidou and Université Paris Descartes, Paris, France; Centre Hospitalier Victor Dupouy d'Argenteuil, Argenteuil, France; Centre Hospitalier Universitaire de Caen, Caen, France; Centre Hospitalier de Châteauroux, Châteauroux, France; Centre Hospitalier Universitaire Jean Minjoz, Besançon, France; Centre Hospitalier de Béthune, Beuvry, France; Centre Hospitalier Universitaire Gabriel Montpied and Université de Clermont-Ferrand, Clermont-Ferrand, France; Centre Hospitalier de Lons-le-Saunier, Lons-le-Saunier, France; and Geneva University Hospital, Geneva, Switzerland.

Acknowledgment: The authors thank the members of the SPEED (Suspected Pulmonary Embolism in Emergency Department) investigators study group and all the physicians and residents of the emergency departments for their invaluable collaboration throughout the study. They also thank Gilles Chatellier, Alain Furber, Franck Verschuren, and Bruno Housset for their help with the protocol and the clinical decision-support system elaboration; the staff of clinical research in Angers, Béatrice Gable, Nathanaëlle Trichereau, Catherine Hue, and Dr. Jean-Christophe Callahan; and the nurses and secretaries of all the centers for their invaluable help. Finally, they thank the Société Française de Médecine d'Urgence for its indubitable scientific help and support.

Grant Support: By a clinical research grant from the National Hospital Clinical Research Project (2004).

Potential Conflicts of Interest: None disclosed.

Reproducible Research Statement:Study protocol: Available (in French) from Dr. Roy (pmroy@chu-angers.fr). Statistical code: Available from Ms. Gillaizeau (florence.gillaizeau@egp.aphp.fr). Data set: Available for purchase from Dr. Roy (pmroy@chu-angers.fr).

Requests for Single Reprints: Pierre-Marie Roy, MD, PhD, Centre Hospitalier Universitaire d'Angers, Service des Urgences, Angers Cedex 9, F-49933, France; e-mail, pmroy@chu-angers.fr.

Current Author Addresses: Drs. Roy and Armand-Perroux: Centre Hospitalier Universitaire d'Angers, Service des Urgences, Angers Cedex 9, F-49933, France.

Dr. Durieux and Ms. Gillaizeau: Hôpital Européen Georges Pompidou, Département d'Informatique Hospitalière, Paris, F-75015, France.

Dr. Legall: Centre Hospitalier Victor Dupouy d'Argenteuil, Service des Urgences, Argenteuil, F-95100, France.

Dr. Martino: Centre Hospitalier de Châteauroux, Service des Urgences, Châteauroux, F-36000, France.

Dr. Hachelaf: Centre Hospitalier Universitaire Jean Minjoz, Service des Urgences, Besançon, F-25000, France.

Dr. Dubart: Centre Hospitalier de Béthune, Service des Urgences, Beuvry, F-62660, France.

Dr. Schmidt: Centre Hospitalier Universitaire Gabriel Montpied, Service des Urgences, Clermont-Ferrand, F-63003, France.

Dr. Cristiano: Centre Hospitalier de Lons-le-Saunier, Service des Urgences, Lons-le-Saunier, F-39016, France.

Mr. Chretien: Centre Hospitalier Universitaire d'Angers, Service de Biostatistiques, Mathématiques et Informatique, Angers Cedex 9, F-49933, France.

Dr. Perrier: Geneva University Hospital and Faculty of Medicine, Division of General Internal Medicine, Department of Internal Medicine, Geneva, CH-1211, Switzerland.

Dr. Meyer: Hôpital Européen Georges Pompidou, Service de Pneumologie et de Soins Intensifs Pneumologiques, Paris, F-75015, France.

Author Contributions: Conception and design: P.M. Roy, P. Durieux, A. Perrier, G. Meyer.

Analysis and interpretation of the data: P.M. Roy, P. Durieux, F. Gillaizeau, J.M. Chretien, A. Perrier, G. Meyer.

Drafting of the article: P.M. Roy, P. Durieux, F. Gillaizeau, J. Schmidt, J.M. Chretien, G. Meyer.

Critical revision of the article for important intellectual content: P. Durieux, A.E. Dubart, J. Schmidt, A. Perrier, G. Meyer.

Final approval of the article: P.M. Roy, P. Durieux, F. Gillaizeau, M. Hachelaf, J. Schmidt, A. Perrier, G. Meyer.

Statistical expertise: F. Gillaizeau.

Provision of study materials or patients: C. Legall, A. Armand-Perroux, M. Cristiano.

Obtaining of funding: P.M. Roy.

Administrative, technical, or logistic support: P.M. Roy, C. Legall, A. Armand-Perroux, J.M. Chretien.

Collection and assembly of data: A.E. Dubart, J. Schmidt, M. Cristiano.

Ann Intern Med. 2009;151(10):677-686. doi:10.7326/0003-4819-151-10-200911170-00003
Text Size: A A A

Background: Testing for pulmonary embolism often differs from that recommended by evidence-based guidelines.

Objective: To assess the effectiveness of a handheld clinical decision-support system to improve the diagnostic work-up of suspected pulmonary embolism among patients in the emergency department.

Design: Cluster randomized trial. Assignment was by random-number table, providers were not blinded, and outcome assessment was automated. (ClinicalTrials.gov registration number: NCT00188032)

Setting: 20 emergency departments in France.

Patients: 1103 and 1768 consecutive outpatients with suspected pulmonary embolism.

Intervention: After a preintervention period involving 20 centers and 1103 patients, in which providers grew accustomed to inputting clinical data into handheld devices and investigators assessed baseline testing, emergency departments were randomly assigned to activation of a decision-support system on the devices (10 centers, 753 patients) or posters and pocket cards that showed validated diagnostic strategies (10 centers, 1015 patients).

Measurements: Appropriateness of diagnostic work-up, defined as any sequence of tests that yielded a posttest probability less than 5% or greater than 85% (primary outcome) or as strict adherence to guideline recommendations (secondary outcome); number of tests per patient (secondary outcome).

Results: The proportion of patients who received appropriate diagnostic work-ups was greater during the trial than in the preintervention period in both groups, but the increase was greater in the computer-based guidelines group (adjusted mean difference in increase, 19.3 percentage points favoring computer-based guidelines [95% CI, 2.9 to 35.6 percentage points]; P = 0.023). Among patients with appropriate work-ups, those in the computer-based guidelines group received slightly fewer tests than did patients in the paper guidelines group (mean tests per patient, 1.76 [SD, 0.98] vs. 2.25 [SD, 1.04]; P < 0.001).

Limitation: The study was not designed to show a difference in the clinical outcomes of patients during follow-up.

Conclusion: A handheld decision-support system improved diagnostic decision making for patients with suspected pulmonary embolism in the emergency department.

Primary Funding Source: French National Hospital Clinical Research Project.


Grahic Jump Location
Appendix Figure 1.
Screenshots from handheld clinical decision-support system.
Grahic Jump Location
Grahic Jump Location
Appendix Figure 2.
Adjusted mean absolute difference in the rate of appropriate diagnostic management between periods, by center.
Grahic Jump Location
Grahic Jump Location
Figure 2.
Flow diagram: 3-month outcomes.

VTE = venous thromboembolism.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Appropriateness Criteria for Diagnosis of Pulmonary Embolism
Posted on November 20, 2009
Benjamin Djulbegovic
University of South Florida & Indiana University
Conflict of Interest: None Declared

Roy at al(1) provided the important contribution to the literature by showing how diagnostic accuracy of pulmonary embolism (PE) can be improved using an handheld decision-support system (CDSS). However, diagnostic accuracy is not the same as the appropriateness criteria. The authors defined PE to be appropriately excluded if the calculated posttest probability of PE was less than 5% while PE was appropriately ruled-in if the posttest probability of PE was greater than 85%. Roy at el do not explain how these criteria were derived. The appropriateness criteria need not only to take into considerations data on the diagnostic tests accuracy but the consequences of management that follow the results of the tests. In case of PE, this means assessing benefits and harms of anticoagulant treatments. We recently showed that according to expected utility theory (EUT) model- the 'gold' criterion of rational decision-making(2)- the threshold below which PE can be ruled out is significantly lower than 5% (often as low as 0.07%!) for most tests used in Roy et al. CDSS. Depending on benefits and harms of anticoagulant treatment, we also showed that PE can be considered ruled-in for non- invasive tests such as spiral CT when the posttest probability exceeds 13- 62%, respectively.(3-5) Therefore, it appears, that in many cases, most rational decision for physicians would be to order a CT angiogram when the probability of PE exceeds 0.07% i.e. the moment PE enters the physician's mind as a differential diagnostic possibility(3-5). Both the current and the previous study by Roy and colleagues(6) demonstrated that vast majority of physicians do not act by ordering diagnostic work-up for PE at the probability less than 1% as rationally indicated by the EUT model.(4) We proposed an alternative explanation- the concept of acceptable regret(3- 5), which can explain both overtesting and undertesting, that dominates the practice of modern medicine. While solution to the 'appropriateness criteria' is not easy, we think that using arbitrary values may not be the most 'appropriate' way to assess physicians' performance.


1. Roy PM, Durieux P, Gillaizeau F, et al. A Computerized Handheld Decision-Support System to Improve Pulmonary Embolism Diagnosis: A Randomized Trial. Ann Intern Med. 2009;151(10):677-686.

2. Bell DE, Raiffa H, Tversky A. Decision making. Descriptive, normative, and prescriptive interactions. Cambridge: Cambridge University Press; 1988.

3. Hozo I, Djulbegovic B. When is diagnostic testing inappropriate or irrational? Acceptable regret approach. Med Decis Making. 2008;28(4):540- 53.

4. Hozo I, Djulbegovic B. Will insistence on practicing medicine according to expected utility theory lead to an increase in diagnostic testing?. . Medical Decision Making 2009;29:320-322.

5. Hozo I, Djulbegovic B. Clarification and corrections of acceptable regret model. Medical Decision Making 2009;29:323-324.

6. Roy P-M, Meyer G, Vielle B, et al. Appropriateness of Diagnostic Management and Outcomes of Suspected Pulmonary Embolism. Ann Intern Med. 2006;144(3):157-164.

Conflict of Interest:

None declared

Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.