Tag Archives: statistics

P-hacking, self-plagiarism concerns plague news-friendly nutrition lab

Photo: Dominic Rooney via Flickr

Some of the most difficult research to make sense of comes from nutrition science. It is difficult, expensive and labor-intensive to conduct randomized controlled trials in nutrition, in part because they require randomizing what people eat and then ensuring they eat what they’re supposed to – no more and no less.

Even when such trials are finished (often at in-patient labs), the populations are usually small and somewhat homogenous, thus reducing the generalizability and overall clinical utility of results. Continue reading

Tip sheet series to focus on red flags to look for in medical studies

With thousands of medical studies published every day, it’s impossible to cover even 1 percent of them. When you can only choose a tiny fraction of studies to cover — particularly if you freelance or your editor gives you some autonomy and flexibility in this area — how do you decide whether or not to cover a study?

Reasons can vary: Some people focus on the better known “more prestigious” journals, although that approach has its drawbacks. Continue reading

Tips help remind reporters to understand limits of the studies we cover

One of the most important skills required of reporters who cover medical research is the ability to find and discuss the limits of the studies we cover.

To that end, a trio of professors at Cambridge University recently published a helpful comment in the journal Nature: “Twenty Tips for Interpreting Scientific Claims.” (If you don’t subscribe, you can read the full article for free here.)

Some of my favorites (in no particular order):

  1. Study relevance limits generalizations – a great reminder that the conditions of any study will limit how its findings can be applied in the real world.
  2. Bias is rife – We talk about several types of bias in the topic section, like reporting bias and healthy user effect. The article reminds us that even the color of a tablet can shade how study participants feel. Continue reading

Stories are waiting to be found in new stats on seniors

Older Americans 2012, a new report from the Federal Interagency Forum on Aging-Related Statistics, is an important resource for reporters on the aging beat.

Core Topics
Health Reform
Other Topics

It’s an overview of all kinds of issues affecting older adults based on data from 2009, the latest available. As such, it doesn’t reflect the full impact of the economic downturn on older Americans. But it’s still full of nuggets of interesting information.

Some items that caught my eye:

  • Fewer seniors are living in poverty. Between 1974 and 2010, the proportion of older adults with incomes below the poverty threshold fell from 15 percent to 9 percent.
  • More seniors now fall in the “high income” category. During the time period specified above, well-off older adults expanded from 18 percent to 31 percent.

Continue reading

Obesity doctor calls journalists’ statistical knowledge into question

Yoni Freedhoff, M.D., founder of Ottawa’s Bariatric Medical Institute, writes about two studies about obesity and questions whether journalists are skilled enough in statistical analysis to accurately report on them.

Freedhoff says a new report refutes an earlier study – published in the New England Journal of Medicine and widely reported by the media – as being statistically flawed. And he is skeptical the new study will receive attention from the journalists who reported the first study.

The original study, “The Spread of Obesity in a Large Social Network over 32 Years” by Nicholas A. Christakis, M.D., Ph.D., M.P.H., and James H. Fowler, Ph.D., was widely reported with headlines proclaiming that “Obesity is socially contagious” in 2007.

A new study by Indiana University’s Russell Lyons, published in Statistics, Politics, and Policy, claims “the assumptions behind the statistical procedures used were insufficiently examined.”

As Freedhoff notes, the NEJM has an impact factor of 50, while Statistics, Politics, and Policy has an impact factor of 0.857, leading one to wonder how many reporters have even heard of the new study.

But Freedhoff – who admits he’s no statistics expert – questions whether journalists will report on the new study because they do not have the statistical knowledge to do so.

All in all, even if you’re not a statistician, Lyons’ paper is worth a sober read and reflection, and here’s something else to chew on – the journalists who were originally all over Christakis’ and Fowler’s work? I’d bet every last penny I’ve got that not a single one of them were skilled enough in statistical analysis to analyze it. Really, why should they have been? They’re journalists, not statisticians. No, instead they smelled a good story, and ran with it. Those same journalists who shouted from the rooftops that obesity’s contagious? I’m betting the vast majority of them are going to be silent on this one, yet wouldn’t re-reporting be the socially responsible, ethical, and journalistic right thing to do?

Update: Brian Reid found this paper, “Examining Dynamic Social Networks and Human Behavior,” that appears to be a response to Lyon’s research – Christakis and Fowler reference his critique specifically at least twice in the paper.

So, reporters, let’s hear what you think: Do you know enough about statistics to analyze and report on the new study? Or were you even aware of the new study?

Covering Medical ResearchIt’s certainly worth pointing to AHCJ’s most recent slim guide here: Covering Medical Research, which helps journalists analyze and write about health and medical research studies.

It offers advice on recognizing and reporting the problems, limitations and backstory of a study, as well as publication biases in medical journals and it includes 10 questions you should answer to produce a meaningful and appropriately skeptical report. This guide, supported by the Robert Wood Johnson Foundation, is a road map to help you do a better job of explaining research results for your audience.

An earlier slim guide, “Covering Obesity: A Guide for Reporters,” also might come in handy for covering the topic.