As policymakers prepare to implement major Medicaid cuts while states receive billions through the Rural Health Transformation Program, journalists will need reliable data to track what happens next.
But for reporters covering rural health, the number in front of you may not tell the whole story. Sometimes, what’s missing from the data may be just as important, Michael Meit, director of the Center for Rural Health and Research at East Tennessee State University, said during an AHCJ webinar.
The problem starts with defining “rural.” There are 72 federal definitions of rural, Meit said, and researchers may use different measures of rurality. Journalists should check how a study defines “rural” before comparing findings or drawing conclusions about rural communities.
Data limitations
The data can also be too limited to tell us much about an individual community. A survey may accurately represent a state or the nation while including too few people from a particular rural county to draw meaningful conclusions.
“You may have three people who’ve responded to a survey that is still representative of the national population, or maybe even the Tennessee population, but you really can’t say much about Washington County, because it was only three people,” Meit said.
Small numbers can also lead to data being withheld to protect people’s privacy. When information is missing, researchers may use statistical models to estimate what is happening. Those estimates can be useful, but journalists need to know when a number is modeled rather than directly measured, Meit said.
“Always drill down to understand the data sources and whether modeling was used,” he said. “If modeling is used, it’s probably still worth reporting, but I would never claim that that is actual data from an individual community.”
Even the strongest data sources have limitations. Mortality data, for example, can take about two years to make their way into federal datasets as records move from counties to states and then to the federal government. In rural communities, stigma can also affect how some deaths are recorded.
“Often those data are underreported if it is something that has stigma attached to it,” Meit said.
‘Rural’ is not a monolith
Rural communities also are far from interchangeable and “rural” should not be treated as a single geographic or demographic category, Meit said. Rural communities differ in geography, demographics and economic conditions, and national comparisons between rural and urban areas can hide those differences.
“At the end of the day, they’re not the same place and we need to know what is actually happening in West Virginia, what is actually happening in Nebraska,” Meit said.
Disappearing data
That makes the current erosion of some data sources especially concerning. Meit pointed to nearly 400 federal datasets that have been deleted or altered and noted that some datasets containing information on race, ethnicity, social drivers of health and other equity measures have been affected. The County Health Rankings, another widely used resource, is also being discontinued.
The answer isn’t for journalists to become statisticians, Meit said. Instead, they should become more careful consumers of data and develop relationships with researchers who can help interpret it.
“We all can’t be experts in this,” he said. Journalists can rely on academic experts, he added, while also learning to examine the data themselves and question how a statistic was produced.
That means looking beyond a ranking or index and finding out what is behind the number.
“Drill into the source data to really understand the data that underpin what the output is because that’s the real story, not necessarily some modeling that gets you to a ranking or an index number,” Meit said.
