JAMA: Don't believe everything you read

[ravi <gadfly@xxxxxxxxxxxx>]

nothing to do with economics, but since this sort of stuff has come
up before on pen-l, i thought some of you might find this interesting:

JAMA: Don't believe everything you read

CHICAGO, Illinois (AP) --One of the world's leading medical journals has
put itself and its competitors under the microscope with research
showing that published studies are sometimes misleading and frequently
fail to mention weaknesses.

Some problems can be traced to biases and conflicts of interest among
peer reviewers, who are outside scientists tapped by journal editors to
help decide whether a research paper should be published, according to
several articles in this week's Journal of the American Medical Association.

Other problems originate in news releases some journals prepare to call
attention to what they believe are newsworthy studies. The releases do
not routinely mention study limitations or industry funding and may
exaggerate the importance of findings, according to one JAMA study.

Wednesday's JAMA, devoted entirely to such issues, "is our attempt to
police ourselves, to question ourselves and to look at better ways to
make sure that we're honest and straightforward and maintain the
integrity of the journals," said Dr. Catherine DeAngelis, JAMA's editor.

The articles "underscore that the findings presented in the press and
medical journals are not always facts or as certain as they seem," said
Rob Logan, director of the Science Journalism Center at the University
of Missouri-Columbia.

DeAngelis said problems are most likely to occur in research funded by
drug companies, which have a vested interest in findings that make their
products look good.

Journal editors are concerned that manufacturers sometimes unduly
influence how researchers report study results, and even suppress
unfavorable findings.

Many top journals require researchers to disclose any ties to drug
companies, and Dr. Jeffrey Drazen, editor of the New England Journal of
Medicine, said editors rely on researchers to be truthful.

"I imagine that from time to time we screw up" and fail to adequately
mention drug company ties, but that is infrequent, Drazen said.

Favoring favorable statistics

One JAMA report found that medical journal studies on new treatments
often use only the most favorable statistic in reporting results, said
author Dr. Jim Nuovo of the University of California at Davis.

His study reviewed 359 studies published between 1989 and 1998 in JAMA,
The New England Journal of Medicine, The Lancet, the British Medical
Journal and Annals of Internal Medicine. Only 26 studies reported
straightforward statistics that clearly assessed the effect on patients.

Most reported only the "relative risk reduction" linked to a specific
treatment, which is the percentage difference between drug-treated
patients and those in a placebo group. That figure is more misleading
than the "absolute risk reduction," which measures the actual difference
between the treatment results compared with the placebo group, Nuovo said.

For example, if 5.1 percent of placebo-treated patients had heart
attacks compared with 3.7 percent of drug patients, the absolute risk
reduction in the drug group would be 1.4 percent. But researchers could
use the relative risk reduction to claim that the drug lowers the risk
of a heart attack 27 percent -- which sounds a lot more impressive.

In another report, researchers from the Veterans Affairs Medical Center
in White River Junction, Vermont, examined 127 news releases from seven
journals: JAMA, The Lancet, Pediatrics, BMJ, Journal of the National
Cancer Institute, Circulation and Annals of Internal Medicine. Few noted
study limitations or drug company funding, said the authors, Drs. Steven
Woloshin and Lisa Schwartz.

Releases were generally prepared by press officers, and the authors said
better editorial oversight could improve the process.

In a third JAMA report, Dr. Richard Horton, The Lancet's editor,
analyzed 10 research articles published in his journal in 2000 and found
that some authors appeared to have censored critical comments from their
co-authors. Disagreements among authors about a study's conclusions
occurred frequently but often were not mentioned in the articles, he said.

Reforming the peer review process could address some problems, said
Fiona Godlee of BioMed Central, an online medical journal publisher that
asks peer reviewers to identify themselves in their reports.

Most print medical journals allow peer reviewers to remain anonymous. In
another JAMA report, Godlee said requiring open review would make
reviewers more accountable and might reveal any conflicts of interest.

Copyright 2002 The Associated Press. All rights reserved. This material
may not be published, broadcast, rewritten, or redistributed.

Find this article at:
http://www.cnn.com/2002/HEALTH/06/04/medical.journals.ap/index.html