Reporter Emily Bazar, of the California Health Care Foundation Center for Health Reporting, endeavored to reveal the meaning behind some surprising statistics: Citizens in a northern California town were five times to six times more likely to undergo elective heart surgery than other Californians. Bazar analyzed the study, conducted by a Stanford professor, to determine if external factors influenced the data, or if Clearlake residents were really receiving unnecessary (and expensive) operations. The center partnered with the San Francisco Chronicle, which published the investigation.
We asked Bazar to share her experiences reporting on this data and to shed light on how journalists can transform statistics into a compelling story. While Bazar’s investigation is based on California data, all or nearly all states collect similar data that can be analyzed. Learn more about such data. Below is what Bazar learned from reporting this story.
Emily Bazar
By Emily Bazar
The rural town of Clearlake sits on the shores of California’s largest freshwater lake, but it is also known for high unemployment, drug use and poverty.
Clearlake is down and out, despite its beautiful surroundings
So, it didn’t make a lot of sense to me that the residents of this isolated community, who often get airlifted to regional hospitals during emergencies, had been undergoing two common heart procedures at exorbitant levels – between five times and six times more than Californians as a whole.
That was the finding of a complex data analysis conducted by Stanford Professor Laurence Baker, who crunched five years (2005-09) of statewide hospital and some outpatient discharge data from California’s Office of Statewide Health Planning and Development, or OSHPD.
(Baker’s research was funded by the nonpartisan California HealthCare Foundation, the same group that funds the CHCF Center for Health Reporting.)
Baker looked at the use of 13 elective procedures – from hip replacements to mastectomies – across the state. He broke the data into 24 established medical geographies known as Hospital Referral Regions and then into smaller Hospital Service Areas.
The largest geographical variations were in elective angioplasty, a procedure used to open blocked arteries that supply blood to the heart, and elective angiography, a diagnostic test that detects coronary artery blockages. He pinpointed high rates of these procedures in broad swaths of California’s Central Valley, but the largest disparities were in Clearlake.
Was this a result of overtreatment, or did Clearlake residents really require more elective heart surgery? I set out to investigate.
I was lucky, because Baker had done the most difficult technical work, including the arduous process of risk-adjusting the rates to level the playing field among California regions. Baker attempted to eliminate differences in the health of populations and their access to health care. More on that later.
There were some data points Baker didn’t provide that I filled in. Baker’s analysis didn’t include the names of hospitals where patients underwent procedures. So, I worked with an OSHPD analyst to replicate Baker’s data run, this time tying the procedures to facilities. OSHPD usually charges researchers for this kind of service, but provides it to journalists for free.
The process was a learning experience and required several iterations before we were able to replicate it, but OSHPD was a willing and patient partner.
The most important thing I learned was that more than 90 percent of Clearlake area residents were going to St. Helena Hospital, at least an hour away, for the procedures. The hospital is part of the Adventist Health network, which also owns the small hospital in Clearlake that doesn’t perform most major procedures or operations.
Knowing this helped focus my reporting.
With the help of my editor, I also turned to the Dartmouth Atlas of Health Care. The Atlas specializes in tracking patterns in Medicare usage, and through its online database, we discovered that in 2005, 2006 and 2007 (the three most recent years for which data are available), Clearlake had the highest inpatient angioplasty rate in the country.
Wow.
This helped bolster Baker’s conclusions and reminded me to use the Dartmouth Atlas more frequently as a reporting tool.
Outside of data reporting, I spent much of my time delving into geographic variation, a phenomenon that has been well-documented by the Dartmouth Atlas, Atul Gawande and others. Simply put, such variations suggest that where you live, and which hospital and doctor you visit, seem to influence how likely you are to go under the knife.
Early on, a source urged me to read “Tracking Medicine,” by John Wennberg, a professor at Dartmouth Medical School and founder of the Dartmouth Atlas. Wennberg’s book is a must for anyone interested in geographic variation, and it drove home some key, seemingly counterintuitive, conclusions.
One is this: Even though hospitals and doctors often blame variation on differences in regional populations – from their health conditions to income levels to access to health care – that generally doesn’t explain the disparities.
Instead, research suggests that other powerful factors are at play, such as differences in how doctors treat diseases, especially when they have discretion in how to treat them.
Wennberg and other experts I interviewed (including Harlan Krumholz at Yale) say doctors’ training and styles often lead them to choose their favorite treatments. That, over time, can create differences in the number of procedures performed across regions.
This is where Baker’s risk adjustments come in.
He adjusted each region’s results for age, sex, race, education, income and health insurance status. He and other researchers say that these demographic and economic factors correlate well with people’s health. For the heart procedures, Baker also used two direct health indicators: rates of heart attack hospitalizations and diabetes diagnoses during hospital stays.
Understanding these adjustments – and their limitations – required some work.
When I approached Adventist officials about the high rates of heart procedures conducted on Clearlake residents, they pointed to the health and socioeconomic status of the population, and flatly rejected the idea that their doctors perform unnecessary procedures. After all, Clearlake residents have high rates of smoking, heart disease and unemployment, they said.
The health professionals questioned Baker’s risk adjustments, saying the rates would have been lower had Baker accounted for factors including hypertension, smoking and the cumulative effect of the area’s poor health and economic status.
I dug into this, and found that even expert statisticians don’t agree on how best to conduct risk adjustments.
John Spertus, a cardiologist and professor at the University of Missouri – Kansas City, who conducts similar research, faulted Baker for not accounting for patients’ level of angina, the chest pain or pressure that is one of the main reasons patients seek angioplasty. “The more severe the symptoms, the more legitimate the procedures,” he said.
But Baker cautioned against using population health as an explanation for extreme variation in procedure frequency, in part because he believes his adjustments addressed health differences caused by regional factors. He and others also pointed out – repeatedly – that there are no places in California where the overall health of the population is five times or six times worse than any other place. To see all of his results click here. His methodology is here.
“The argument that these vast differences could be explained by some environmental or other factor just runs counter to all the history of these massive studies done on procedure variations,” Wennberg said. “It’s just not possible.”
Spertus, too, acknowledged that even though Baker didn’t account for angina, “the variation is never that large” as it is between Clearlake residents and other Californians.
It seemed that many experts believed Clearlake’s heart procedure frequency was indeed anomalous.
I did some shoe-leather reporting to see if Baker’s results held up on the ground:
- I hit the senior centers in Clearlake and found residents concerned about cardiac procedures they underwent.
- I canvassed doctors in the area, who also discussed concerns with St. Helena’s and certain local cardiologists’ practices.
- St. Helena officials provided additional data showing that the number of elective angioplasties the hospital performed among Clearlake residents declined by 60 percent over the five years of the study.
Despite some dissent, Baker’s analysis raised some important questions about the the number of heart procedures performed on Clearlake residents. I am working on a follow-up story based on tips we received after the stories ran, and am applying lessons I learned in the first round of reporting to the upcoming piece.
Emily Bazar is a senior writer for the California HealthCare Foundation Center for Health Reporting.
