State health inspections are the heart of the government’s five-star rating system for nursing homes. But, not everything inspectors find makes it into the ratings formula, a New York Times investigation found.
Thousands of violations are downgraded or deleted altogether during a secretive appeals process. Other serious transgressions are excluded from public view because they were uncovered by federal and not state inspectors. Some are missing because of a technical glitch, a spokesman for the Centers for Medicare and Medicaid Services confirmed.
This finding came as a surprise to us [colleagues Katie Thomas and Jessica Silver-Greenberg]. We had been writing a series about the “Care Compare” rating system and knew from earlier reporting that there were some well-known issues with inspections. Eldercare advocates have long lamented that many violations that harm residents are classified as having caused “potential harm.” In one of our earlier stories, we mined the text of inspection reports and identified dozens of incidents that would be viewed as serious sex crimes outside of nursing homes but were treated as minor offenses by the ratings system.
We also knew that the enforcement system varied significantly by state. The ratings formula attempted to account for such discrepancies.
But we had no idea the extent to which conduct uncovered by inspectors was being quashed until a source suggested we look into the most opaque appeals process. Not only is there no easy way for residents and families to find out if a nursing home is appealing an individual violation, there is also no national accounting of how many violations are appealed, nor a tally of how many cases are overturned or modified.
We set out to shine some light on the topic, taking a multi-pronged approach. We began contacting health departments in all fifty states and asked them to provide an accounting of their “Informal Dispute Resolution” process.
The IDR system is run at the state level but within federal guidelines. We thought that most states would easily be able to tell us, in aggregate, how many appeals were heard and to provide us with a summary of the outcomes.
This scenario happened in some states, but many were hesitant and required us to file a FOIA request. Other states refused altogether. Some claimed that CMS told them they weren’t allowed to release any information. One state complied not by giving us statistics, but a 120-page PDF file listing individual hearings. I had to write code to go through the document and count cases and outcomes. Another state sat on our FOIA for more than a month and finally released data that made no sense.
We used a spreadsheet to keep track of our requests and the outcomes. Once information began trickling in, my job as the data reporter on the project was to standardize the responses. We asked states to tell us how many violations had been appealed since 2016, how many were upheld and how many were modified or deleted. Some states provided us just that, but others provided eight or nine different outcomes, some of which had to be collapsed into the three main categories.
In the end, we used information obtained from 18 states to report that more than 2,700 violations had been deleted since 2016, a figure that would have grown had more states complied.
Meanwhile, we noticed another quirk. Our original source had noted how there was also a federal appeals process that was nearly as opaque. She told us about the relatively obscure Health and Human Services Departmental Appeals Board, and a website that posted decisions written by DAB administrative law judges.
These decisions cover a wide range of topics but include appeals filed by nursing homes seeking to overturn citations and fines. We identified the nursing home cases and created another spreadsheet to capture information about each case. We then set about determining whether these cases were part of Care Compare.
This was tricky to measure since some of these federal appeals involved violations four or more years old, and Care Compare only includes results from roughly the last three years of inspections.
Fortunately, CMS makes available an archive of the database that powers the site, and we were able to download the monthly data files going back to 2014. I wrote some code that looped through these files and generated a database of every violation that ever appeared on Care Compare, including when the violation first appeared and when it was removed.
From this database, we were able to identify ten examples of serious violations that were upheld by the federal judge, but that had never become public via Care Compare. Sometimes, we learned, entire inspection reports were missing. Other times, CMS had posted inspection reports but redacted the most serious violations.
We found even more examples. Some colleagues had gathered a set of serious nursing home violations from 2020 while pursuing a different story. They had graciously offered to share these documents with us since they knew we were reporting on nursing homes in 2021.
Out of curiosity, I pulled one of the reports from April of 2020 — it was an infection control violation from Arkansas. When I jumped onto the Care Compare site, the violation was nowhere to be found, and a consumer in 2021 would have no idea of knowing that a state inspector had placed the facility in “immediate jeopardy” the year before.
We also noticed when looking through documents that there were reports written by federal inspectors. CMS conducts hundreds of inspections every year to audit the performance of state inspectors. But it turns out the results of these inspections are, per CMS policy, never made public, even if the inspectors find serious violations, and we found three examples of those.
Some sources told us that the five-star system generally correlates with the quality of facilities. But a general correlation across thousands of homes doesn’t help a consumer trying to vet a specific facility. If the data is wrong or misleading for that particular establishment, the results can be life-altering. So, it felt pretty important to get this story told.
Robert Gebeloff is a reporter specializing in data analysis. He works on in-depth stories where numbers help augment traditional reporting.
