Yoni Freedhoff, M.D., founder of Ottawa’s Bariatric Medical Institute, writes about two studies about obesity and questions whether journalists are skilled enough in statistical analysis to accurately report on them.
Freedhoff says a new report refutes an earlier study – published in the New England Journal of Medicine and widely reported by the media – as being statistically flawed. And he is skeptical the new study will receive attention from the journalists who reported the first study.
The original study, “The Spread of Obesity in a Large Social Network over 32 Years” by Nicholas A. Christakis, M.D., Ph.D., M.P.H., and James H. Fowler, Ph.D., was widely reported with headlines proclaiming that “Obesity is socially contagious” in 2007.
A new study by Indiana University’s Russell Lyons, published in Statistics, Politics, and Policy, claims “the assumptions behind the statistical procedures used were insufficiently examined.”
As Freedhoff notes, the NEJM has an impact factor of 50, while Statistics, Politics, and Policy has an impact factor of 0.857, leading one to wonder how many reporters have even heard of the new study.
But Freedhoff – who admits he’s no statistics expert – questions whether journalists will report on the new study because they do not have the statistical knowledge to do so.
All in all, even if you’re not a statistician, Lyons’ paper is worth a sober read and reflection, and here’s something else to chew on – the journalists who were originally all over Christakis’ and Fowler’s work? I’d bet every last penny I’ve got that not a single one of them were skilled enough in statistical analysis to analyze it. Really, why should they have been? They’re journalists, not statisticians. No, instead they smelled a good story, and ran with it. Those same journalists who shouted from the rooftops that obesity’s contagious? I’m betting the vast majority of them are going to be silent on this one, yet wouldn’t re-reporting be the socially responsible, ethical, and journalistic right thing to do?
Update: Brian Reid found this paper, “Examining Dynamic Social Networks and Human Behavior,” that appears to be a response to Lyon’s research – Christakis and Fowler reference his critique specifically at least twice in the paper.
So, reporters, let’s hear what you think: Do you know enough about statistics to analyze and report on the new study? Or were you even aware of the new study?
It’s certainly worth pointing to AHCJ’s most recent slim guide here: Covering Medical Research, which helps journalists analyze and write about health and medical research studies.
It offers advice on recognizing and reporting the problems, limitations and backstory of a study, as well as publication biases in medical journals and it includes 10 questions you should answer to produce a meaningful and appropriately skeptical report. This guide, supported by the Robert Wood Johnson Foundation, is a road map to help you do a better job of explaining research results for your audience.
An earlier slim guide, “Covering Obesity: A Guide for Reporters,” also might come in handy for covering the topic.
