Researchers agree: A study is a terrible thing to waste

About Brenda Goodman

Brenda Goodman (@GoodmanBrenda), an Atlanta-based freelancer, is AHCJ’s topic leader on medical studies, curating related material at healthjournalism.org. She welcomes questions and suggestions on medical study resources and tip sheets at brenda@healthjournalism.org.

Spend any significant amount of time reporting on research and you’re bound to run across a real stinker of a study.

Too often, the studies that become clickbait on the web or turn up in women’s magazines – usually boiled down to a surprising health tip – are just, well, how do I put this? Crap.

There are a lot of those kinds of studies in the world. Studies that are too small to be meaningful, or they ask bad or useless questions, they’re poorly designed or they essentially answer a question that’s already been repeatedly answered.

These kinds of studies exist because the publish-or-perish culture of academia rewards volume over value. And let’s accept our part in this, too. There’s always a media outlet that’s willing to trumpet a surprising, if completely unsound, study.

In a microcosm, a bad study or two can raise an eyebrow or a chuckle. In a macrocosm, however, the situation is dire.

In a 2009 viewpoint published in The Lancet, Sir Iain Chalmers, one of the founders of the Cochrane Collaboration, and Paul Glasziou, M.B.B.S., a professor of evidence-based medicine at Bond University, in Australia, estimated that 85 percent of research investment is wasted. That’s 85 percent of a $240 billion global biomedical research budget and the time of millions of people who volunteer to participate in clinical trials.

Today, The Lancet is building on that viewpoint by publishing an important series of papers that discuss ways to curb waste in research. The papers, from leading research thinkers, focus on five areas of improvement.

  1. They call on scientists and funders to set out to answer better, more relevant research questions. One of the best ways to do this, Chalmers says, is by first doing a systematic review of existing evidence to find out what’s already been done.
  2. Stanford’s John Ioannidis and colleagues ask researchers to improve the design, conduct, and analysis of studies. Just the abstract of this paper is a terrific distillation of the problems that haunt medical research.
  3. Professor Rustam Al-Shahi Salman, of the University of Edinburgh in the United Kingdom, and colleagues ask regulators to make the management of clinical trials less burdensome.
  4. Dr. An-Wen Chan, of the University of Toronto in Toronto, Canada, and colleagues call on researchers and journals to make study protocols, results, and datasets more freely available.
  5. Professor Paul Glasziou, of Bond University in Robina, Australia, and colleagues call on researchers and journals to improve the reporting of the research results.

Let’s hope researchers and policy makers take these strategies to heart. Think about what could be accomplished (and how exciting it would be to cover medical studies) if we got more bang for our scientific buck.

3 thoughts on “Researchers agree: A study is a terrible thing to waste

  1. Robert Matthews

    Good summary.

    You’re right that the silly stories about the dangers/benefits of (say) coffee based on a study of 20 students hide the much greater scandal of wasted research. But it’s not just the consequent waste of research funds; far more egregious, to my mind, is the human impact.

    Most scandalous of all is the human impact of redundant trials – ie trials of therapies which a systematic review would have shown to be pointless, as efficacy/failure had already been established by previous trials.

    Clearly, setting up an expensive randomised controlled trial of a therapy already known to be effective or useless is a waste of research funding. But the human cost is shocking. Redundant trials of therapies known to be effective means half the patients in the trial (eg, those in the placebo arm) are being denied a proven therapy. The flip-side also holds, with redundant trials of useless/dangerous therapies unnecessarily exposing patients to risk.

    The classic case of this centres on aprotinin (see eg http://www.ethicshare.org/node/475942 ), where tens of thousands of patients were unnecessarily exposed to risk. One of the slides put up by Iain Chalmers yesterday showed another,concerning colorectal surgery. A quick totting up of the patients in redundant trials wrongly denied appropriate treatment gave a figure of around 5,000.

    So that’s another 5,000 patients who could make a good claim to have been subjected to medical negligence. I wonder if there are any attorneys who might have a view on this ? A big class action might be one way of getting researchers to do a systematic review of what’s already known before recruiting patients…

  2. Pingback: It's only an illusion - Page 14 - Christian Forums

Leave a Reply