Steven Woloshin, MD, MS; Lisa M. Schwartz, MD, MS
Woloshin S, Schwartz LM. Communicating Data About the Benefits and Harms of Treatment: A Randomized Trial. Ann Intern Med. 2011;155:87-96. doi: 10.7326/0003-4819-155-2-201107190-00004
Download citation file:
Published: Ann Intern Med. 2011;155(2):87-96.
Despite limited evidence, it is often asserted that natural frequencies (for example, 2 in 1000) are the best way to communicate absolute risks.
To compare comprehension of treatment benefit and harm when absolute risks are presented as natural frequencies, percents, or both.
Parallel-group randomized trial with central allocation and masking of investigators to group assignment, conducted through an Internet survey in September 2009. (ClinicalTrials.gov registration number: NCT00950014)
National sample of U.S. adults randomly selected from a professional survey firm's research panel of about 30 000 households.
2944 adults aged 18 years or older (all with complete follow-up).
Tables presenting absolute risks in 1 of 5 numeric formats: natural frequency (x in 1000), variable frequency (x in 100, x in 1000, or x in 10 000, as needed to keep the numerator >1), percent, percent plus natural frequency, or percent plus variable frequency.
Comprehension as assessed by 18 questions (primary outcome) and judgment of treatment benefit and harm.
The average number of comprehension questions answered correctly was lowest in the variable frequency group and highest in the percent group (13.1 vs. 13.8; difference, 0.7 [95% CI, 0.3 to 1.1]). The proportion of participants who “passed” the comprehension test (≥13 correct answers) was lowest in the natural and variable frequency groups and highest in the percent group (68% vs. 73%; difference, 5 percentage points [CI, 0 to 10 percentage points]). The largest format effect was seen for the 2 questions about absolute differences: the proportion correct in the natural frequency versus percent groups was 43% versus 72% (P < 0.001) and 73% versus 87% (P < 0.001).
Even when data were presented in the percent format, one third of participants failed the comprehension test.
Natural frequencies are not the best format for communicating the absolute benefits and harms of treatment. The more succinct percent format resulted in better comprehension: Comprehension was slightly better overall and notably better for absolute differences.
Attorney General Consumer and Prescriber Education grant program, the Robert Wood Johnson Pioneer Program, and the National Cancer Institute.
Optimal ways of communicating the potential benefits and harms of treatments are not clear.
This randomized trial involving 2944 adults compared 5 numeric formats for presenting outcomes that could occur with drug treatment. People seemed to best understand information that was presented as a simple percentage. Even with the percent format, however, about one third of participants had difficulty understanding data about benefit and harm risks.
The researchers studied adults who agreed to participate in a survey about 2 hypothetical drug treatments rather than patients facing actual medical decisions.
Presenting probable treatment outcomes in a simple percentage format might improve comprehension.
The percent plus natural frequency and variable frequency–alone format groups are not shown but can be constructed on the basis of the numbers in the figure.
Natural frequency: Absolute risks and differences were expressed as whole numbers per 1000 (for example, 20 in 1000). Because natural frequencies are the most commonly recommended format, we used them as the reference group.
Variable frequency: Absolute risks and differences were expressed as frequencies in which the denominator was adjusted so that it is the smallest multiple of 10 necessary to keep the numerator greater than 1. For example, 2% is expressed as 2 in 100, 0.2% is 2 in 1000, and 0.02% is 2 in 10 000. To minimize confusion, denominators varied only between table rows.
Percent: Absolute risks and differences were expressed as percents rounded to whole numbers, unless decimals were needed to see the absolute difference (for example, 3.3% [placebo group] vs. 2.5% [drug group] = 0.8%).
Percent plus natural frequency: Absolute risks were expressed as both percent and a natural frequency (x in 1000). To avoid data overload, absolute differences were expressed only as percents.
Percent plus variable frequency: Absolute risks were expressed as both percent and a variable frequency (as explained above). To avoid data overload, absolute differences were expressed as percents.
There were 2944 participants overall and 1037 with low numeracy. “Low numeracy” is defined as answering 0 or 1 of the 3 numeracy questions correctly. The error bars represent the upper bound of the 95% CI.
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
David A. Nardone
Oregon Health & Sciences University
July 27, 2011
Choice of Methods in Presenting Scenarios to Patients
After reading the article by Woloshin and Schwartz (1) on communicating risk, I reviewed a similar publication by the same authors (2). I do not disagree with the findings in either study, but I believe the problem is the choice of methods in presenting scenarios to patients. The key is to use concrete and personalized images that give meaning to abstract concepts of benefit and harm (3). Begin with using an appropriate frame of reference. If the denominator is 100, use the number of US Senators; if 1000, cite the number of soldiers in a battalion; if 10,000 use the capacity of the local AAA baseball team; if 1000,000 refer to the attendance at the annual OSU/Michigan football game. To review comparative data, ask the patient to imagine two separate barrels of ping-pong balls. One barrel is marked with the intervention (Questor) and the other placebo. In the Questor barrel, 25 ping-pong balls are red (heart attack), and 975 are white (no heart attack). For the placebo, 33 ping-pong balls are red and 967 white. Repeat the scenarios for side effects. Finally, ask the patient to imagine they are blindfolded and are reaching into each barrel and request they assess how much risk they are willing to assume to achieve the stated benefit. For primers in this area, the comparative scenarios can be illustrated in drawings.
David A. Nardone, MD (Retired) Clinical Director Primary Care Staff Physician VHA Medical Center Professor Emeritus Oregon Health & Sciences University 6714 NE Copper Beech Drive Hillsboro, OR 97124-5094
1. Woloshin S, Schwartz LM. Communicating Data about the Benefits and Harms of Treatment: A Randomized Trial. Ann Intern Med 2011; 155: 87-96.
2. Woloshin S, Schwartz, LM, Welch G. The Effectiveness of a Primer to Help People Understand Risk. Ann Intern Med 2007; 146: 256-265.
3. Covello VT, Sandman PM, Slovic P. Risk Communication, Risk Statistics, and Risk Comparisons: A Manual for Plant Mangers. Chemical Manufactuers, 1988, Washington, DC, page 15.
VA Outcomes Group, White River Jct., VT and the Dartmouth Institute for Health Policy and Clinical P
October 4, 2011
Dr. Nardone's suggestion is intended to help people develop a sense of numbers using concrete images.
For this technique to work, though, it would be important to use images familiar to the target audience. We doubt that most Americans know how many soldiers are in a batallion (we don't)? Or the capacity of the field for the local AAA baseball team (we don't even know if we have a AAA team)? Or even the number of US Senators (we knew this one)?
Even with familiar images, though, there may still be problems. Changing the denominators to accommodate chances of different magnitude may undermine communication. In our trial, people had the most trouble understandingthe variable frequency format where denominators changed by orders of magnitude (e.g. 100, 1,000, 10,000).
While Dr. Nardone's approach may be useful to teach concepts, it would be not be feasible for the kinds of applications we envision: efficiently summarizing the multiple benefits and harms of medical interventions.
Steven Woloshin, MD, MS and Lisa M. Schwartz, MD, MS
VA Outcomes Group, White River Jct., VT and the Dartmouth Institute for Health Policy and Clinical Practice
Results provided by:
Copyright © 2017 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only