Laurie Barclay, MD

June 7, 2002 — When interpreting results of clinical trials, doctors should beware of misleading data presentation, suggests an article in the June 5 issue of The Journal of the American Medical Association. Too often, research papers and news reports present only the relative risk reduction, which may exaggerate an apparent drug effect unless compared with the absolute risk reduction or number needed to treat.

"No one number can ever be relied on exclusively to interpret the results of a study," lead author Jim Nuovo, MD, from the University of California, Davis, tells WebMD. "If you give just a relative risk reduction, that is not a good way of doing this. You can't just use one value. It is extremely important to put all these numbers into context."

Despite recommendations by the Consolidated Standards of Reporting Trials (CONSORT) to improve the quality of reporting for randomized, controlled trials, Nuovo's group found that few authors in frequently cited peer-reviewed journals expressed their findings in terms of absolute risk reduction or number needed to treat. They reviewed randomized, controlled trials showing a significant drug effect reported in the Annals of Internal Medicine, British Medical Journal, The Journal of the American Medical Association, The Lancet, and the New England Journal of Medicine in 1989, 1992, 1995 and 1998.

Of 359 eligible articles, the authors reported number needed to treat in only 8 articles, of which 6 were from 1998, and absolute risk reduction in only 18 articles, of which 10 were from 1998.

Nuovo cited an example of how relative risk reduction can be misleading out of context. If 5% of patients receiving aspirin but only 4% of patients receiving a new drug had a cardiovascular event over five years, relative risk reduction would be 20%, but absolute risk reduction would only be 1%: "With a 20% reduction, why would anybody take aspirin? But [if] there was only a 1 in 100 difference in risk, now how would you feel about it?"

In a separate article in the same issue, Steven Woloshin, MD, and Lisa M. Schwartz, MD, of Dartmouth Medical School conclude that medical news releases may also exaggerate drug effects, minimize study limitations, and fail to mention industry funding.

"The public and many physicians often learn about new medical research through the news media, rather than medical journals," they write. "We think that journals can and should do more to enhance the quality of medical reporting."

In January 2001, the authors interviewed press officers at nine prominent medical journals and analyzed 127 news releases for the 6 issues of each journal preceding the interviews. Seven of the nine journals routinely issued news releases for articles selected by the editor and press office based on perceived newsworthiness. Although editorial input varied from none to intense, there were no standards for acknowledging limitations or for data presentation.

Of the 127 releases analyzed, 29 (23%) noted study limitations, and 83 (65%) reported main effects using numbers. Of 58 releases reporting differences between study groups, 26 (55%) reported the corresponding base rate, the statistic least likely to exaggerate drug effects. Of 23 releases describing studies receiving industry funding, only 22% acknowledged this funding.

As difficult as it is for physicians to wade through the vast morass of clinical trial information, it is even more perilous for patients who rely on what their doctors tell them to make informed decisions. Mark V. Williams, MD, director of hospital medicine at Atlanta's Emory University, explains that many physicians are uncomfortable with concepts of data analysis.

"This is a very complicated issue," he tells WebMD. "You have to weigh the good against the harm. I personally favor telling patients about the absolute risk reduction, and I also talk about what is the cost of that."

Duke University psychologist Ruth S. Day, PhD, agrees: "One of the ways to communicate the risk is not to overload the communication with all the benefits. All kinds of techniques are used to report clinical trials in ways that make it easier to understand the benefits but not the risks. Let's make it equally easy to understand the benefits and the risks."

JAMA. 2002;287(21):2813-2814, 2856-2858

Reviewed by Gary D. Vogin, MD