Journalists should learn about study design, evidence-based medicine

Joanne Kenen

About Joanne Kenen

Joanne Kenen, (@JoanneKenen) the health editor at Politico, is AHCJ’s topic leader on health reform and curates related material at healthjournalism.org. She welcomes questions and suggestions on health reform resources and tip sheets at joanne@healthjournalism.org. Follow her on Facebook.

Earlier this month I saw on Twitter one of those collisions between journalism and wonkdom. Maybe “collision” isn’t the right word; maybe it was some kind of interspecies mating dance. Anyhow, the gist of it was that we, journalists, don’t know how to evaluate evidence and someone should step in and teach us.

So I stuck in my two cents (or, rather, my two tweets) pointing out that, yes, there is a need for training and, yes, there are places to get the training, including through AHCJ. (See more after the Twitter discussion.)

So, before I remind you about those resources, just a word on why we need them:

On the surface, it may seem that AHCJ houses two kinds of health journalists – those of us who report on the science side of things, and those of us who are more in a policy world. But some of us do both – and research/evidence/evaluating science are also becoming an increasing part of the underpinnings of policy beats. Value-based purchasing, comparative effectiveness, benefits of screening/prevention, quality measures, outcome research … these are all part of the health care reform story.

That doesn’t mean all of us must become  economists/biologists/epidemiologists/statisticians. Old fashioned reporting – including calling experts who can help us make sense of numbers  – is certainly part of the job. But it’s also good to have some sense of what the experts are talking about, what these numbers mean. Why a study on N=16 patients doesn’t really tell us that much. What do we mean by “endpoints,” “outcomes,” “progression?” What’s relative versus absolute risk? Etc.

So for those of you who haven’t  taken a cyber-stroll through the AHCJ website, take five minutes and check out tip sheets, resources and slim guides. Of particular relevance to this discussion is Gary Schwitzer’s slim guide, “Covering Medical Research.”  There’s also a tip sheet/PDF presentation by Schwitzer on “Understanding studies.” His Health News Watchdog blog is also useful.

Reporting on Health (at USC) also has a lot of useful resources, and this essay “Tricks of the Trade: Finding Nuggets in the River of Medical Studies” is a good entry point to understanding data. It’s by Lauran Neergard, a longtime Associated Press health and science writer.

In addition,  there’s a course called Medicine in the Media, sponsored by National Institute of Health’s Office of Medical Applications of Research.  It’s free, but you have to apply, and there’s not room for everyone.  I know of at least one recent summer (the only one I, personally, could have managed the timing!) it wasn’t given, and as of now, there’s nothing on the website about this year. But you can sign up for email notifications, so if you are interested, do that now because the deadline in past years has been early.

The Poynter Institute has some online modules, too. Lots of the focus is on new media and writing and story telling, but there is a math basics refresher for those of you who haven’t taken it since the SATs, some online Excel training, and a unit on reporting on nonprofits.

An aside –  another kind of data to watch out for: Polls. At the AHCJ conference in Philadelphia last year, we had a session on understanding political polls. Here are materials from that presentation, from Claudia Deane, associate director of public opinion and survey research, Kaiser Family Foundation, and representing the American Association for Public Opinion Research.

The basics are useful not just for political polls but for all those other polls that end up in health reporters’ inboxes. An awful lot of them are meaningless, either because of the sampling methodology (or lack thereof) or because the questions were written in such an apple pie, no-trade-offs way. Americans love pain control!  Americans want to cure cancer! Americans like healthy children! An online survey of doctors who may or may not be in a relevant specialty but who felt like clicking on the poll this week agree  with X, Y and Z.

We’ll take a look at other resources out there in future posts.

And one word of wisdom. Get familiar with these concepts and materials, but don’t necessarily try to cram your head overly full of information or skills  that you aren’t going to put to use right away.  When I was a Kaiser Media Fellow a few years ago, the program included three days of pretty intense Excel training. But my reporting project wasn’t data driven.  Having that basic familiarity with Excel is useful and it gives me a basis to build on if and when I need to. But, for me, a three-day crash course – without immediate application – was probably overkill. (Although we had a lot of good meals in St. Pete!)

[Editor’s note: If you’re looking for Excel help, check out AHCJ’s tip sheet “Intro to investigating health data using spreadsheets” or, for more advanced help, “Finding patterns and trends in health data: Pivot tables in spreadsheets.” Of course, there will be training in all of these topics and more at Health Journalism 2012.]

Joanne KenenJoanne Kenen (@JoanneKenen) is AHCJ’s health reform topic leader. If you have questions or suggestions for future resources, please send them to joanne@healthjournalism.org.