Duane Schrag of the Salina (Kan.) Journal recently discovered critical data was missing from the federal Nursing Home Compare data online. The federal government encourages consumers to use Nursing Home Compare to help them choose long-term care facilities. It takes into account variables such as health inspection results, nursing home staff data, quality measures and fire safety inspections.
Additionally, reporters have used the data to investigate nursing homes. For example, Milwaukee Journal-Sentinel reporters Mary Zahn and Ben Poston used it in their two-part examination of problems in Wisconsin’s nursing homes. Nursing Home Compare appears to be one of many sources used in a series of stories by Lisa Chedekel, Lynne Tuohy and Christopher Keating of the Hartford Courant about deficient care in one of Connecticut’s most prominent nursing home chains.
In this article, Schrag shares how he discovered the holes in the data and what he learned about Nursing Home Compare.
By Duane Schrag, Salina (Kan.) Journal
I stumbled onto the story when a co-worker wrote a story that said a nursing home in our readership area would be closing soon and quoted an official as saying that if the home remained open it would have to spend $100,000 upgrading its fire suppression sprinkler system. Because of a recent AHCJ workshop, I knew that fire code deficiencies are contained in the Nursing Home Compare reports.
However, when I looked up the nursing home in Nursing Home Compare, it showed no fire code violations in recent years. I mentioned this to the reporter, suggesting that perhaps the fire code issue was being offered as an excuse (it’s a familiar complaint – “Government regulations forced me out of business.”). But the reporter talked to the state fire marshal, who confirmed that the nursing home had in fact been given a deadline to correct sprinkler system deficiencies.
So I made a follow-up call to Centers for Medicare & Medicaid Services in Kansas City and was given a rather vague explanation for why the deficiency didn’t show up in Nursing Home Compare. I wasn’t convinced, but I didn’t immediately follow through. Bad response – if at all possible, ALWAYS get an answer that truly passes a straight-face test.
Several months later I was at an AHCJ Midwest Health Journalism Fellowship workshop and I brought up this case during a brainstorming session on rural health issues. Jeff Porter, the organization’s special projects director, said he really thought the story deserved aother look. He was right.
To do that, I needed not only a lot of data but also a solid grasp of the process – how inspection information eventually ends up on the Nursing Home Compare Web site.
In a nutshell:
CMS contracts with states to do the annual and complaint-initiated inspections of nursing homes. These inspections are divided into two categories – quality of care and life safety. In Kansas, the Kansas Department on Aging takes care of the quality of care inspections but it sub-contracts the life safety inspections to the state fire marshal’s office.
Inspection results are keyed into a CMS-managed database using a system called ASPEN. The Kansas Department on Aging and the state fire marshal’s office input their results separately, but the state fire marshal supplies copies of its inspections to the Department on Aging, since that department is officially responsible for the inspections.
So the first thing I did was ask the Department on Aging for the fire inspection reports. They showed there had been deficiencies at the nursing home before it closed. With the hard copies in hand I called the fire marshal’s office. The office had no idea why the deficiencies didn’t appear in Nursing Home Compare, but insisted they had been keyed into ASPEN.
Nursing Home Compare pulls its information out of ASPEN.
With that information, I went back to CMS. It was only because I had hard copies of the inspections in my hand, as well as the on-the-record statements by the fire marshal’s office, that I was able to demand an explanation. I think it would be hard to exaggerate how squishy CMS was in its responses: I was given many answers that kinda, sorta explained it but really did not.
For what it’s worth, I suspect CMS at this point wasn’t aware of this particular problem. If CMS officials were, their capacity for obfuscation is formidable.
Fairly early on in the talks I learned that fire code deficiencies can be waived. “Waived” is a bit misleading; it really means the home is given more than the automatic minimal time frame to correct the problem. Initially CMS tried to tell me that the reason the deficiencies didn’t show up was because they had been waived (given extra time to correct) and, since the nursing home planned to close, were never put into the computer system.
This, of course, isn’t what the fire marshal had said. That office maintained it had keyed the deficiencies in. In addition, I had the hard copies of the forms and they showed that, while some deficiencies had been waived, others had not.
But verifying that none of the deficiencies appeared in Nursing Home Compare became a problem: Because the home was closed, none of its inspection data showed up in the data set. So I went online and found the archived inspection information, which goes back to the third quarter of 2003. That confirmed that the nursing home had been cited several times in the past and their violations had shown up in Nursing Home Compare.
Furthermore, even though the hard copies showed some of these older violations had been waived, all showed up in Nursing Home Compare prior to 2007. Since then, none of the violations had shown up in Nursing Home Compare.
It was only after I had this level of information that I was able to get CMS to explain what happened. And even at that, there was a last-ditch effort to avoid giving the explanation. At the beginning of the conference call in which the reason was finally provided, a CMS official rather pointedly insisted that CMS already had answered all my questions and suggested that I was just wasting everyone’s time. It was only after I countered that, in fact, the central question still remained that I finally got the answer.
All code violations are keyed into ASPEN, whether they are waived or not. But if they are waived, the waiver status is set as either “temporary” or “continuing.” It turns out that if any one violation is given “temporary” waiver status, the Nursing Home Compare software wouldn’t report any fire code violations at that facility. It was simply a software bug.
Even at that, CMS was reluctant to be entirely candid. CMS initially said that the Kansas fire marshal’s office was “miscoding” the waivers. It could very well be that some waivers that should have been “continuing” were being entered as “temporary,” but the truth is that it was because the waiver was flagged as “temporary” – rightly or wrongly – that Nursing Home Compare was suppressing it.
CMS did mention that the problem may have affected data from other states. It was then I realized I could handily determine how widespread the problem was.
Recall that the archived dataset is for three years. I waited until a week after the “fix” was applied and then downloaded the latest Nursing Home Compare dataset. That provided a new snapshot that went back to mid-2006. I then checked to see how many 2007 and 2008 violations appeared in the latest dataset but were missing from the 2008 Q4 archived dataset.
Each of the annual files contains quarterly snapshots of the Nursing Home Compare data. The oldest I could find were the FY2003 data (quarters 3 and 4), which gives you a view that reaches back to 2001.
One of the first issues I had to deal with was the amount of data. Initially I really didn’t need inspection data from other states, so I extracted just the Kansas nursing home records. I wrote a small routine in Access VisualBasic that read through each table and wrote the Kansas nursing home records to a tab-delimited text file. That gave me a data set that my laptop could manage with acceptable speed.
Once the data was manageable, I found that the Nursing Home Compare software blocked more than 1,000 fire code violation reports involving Kansas nursing homes in 2006, a similar number in 2007 and almost 800 in the first six months of 2008. Nationally, about 21,600 reports were blocked during that same period.
