New data set helps reporter pinpoint critical staffing concerns in skilled nursing facilities Date: 07/25/18
In April, Medicare began using data to rate staffing for more than 14,000 skilled nursing facilities. Data from the Payroll-Based Journal provides a much better look at the how staffing relates to the quality of care than the less precise — and too easy to inflate — staffing data Medicare had been using since 2008, which was based on two-week snapshots of staffing homes provided to inspectors. The data shows staffing and occupancy on every day. It’s an unprecedented degree of granularity that allows for new levels of inquiry.
Jordan Rau of Kaiser Health News used the new data to focus on the wide fluctuations in nursing home staffing from day to day, with staffing consistently plummeting on weekends. Here he discusses some of his methodology and how he made some decisions about how to use the data.
By Jordan Rau
I was asked not long after the story ran what lessons had I learned when tackling a massive new CMS data set that led to a recent front-page story in The New York Times.
Well, for one thing, Centers for Medicare & Medicaid Services is not known for linguistic playfulness. Yet, someone there must have been chuckling when they named its rich new data source for nursing home staffing levels the Payroll-Based Journal — or PBJ.
Like that classic sandwich, the PBJ data set is irresistible. CMS created it to fulfill a requirement of the Affordable Care Act to improve the accuracy of its five-star staffing ratings on Medicare’s Nursing Home Compare website. The dataset contains payroll records that nursing homes are required to submit to the government.
In April, Medicare began using the data to rate staffing for more than 14,000 skilled nursing facilities (SNFs). PBJ data provides a much better look at the how staffing relates to the quality of care than the less precise — and too easy to inflate — staffing data Medicare had been using since 2008, which was based on two-week snapshots of staffing homes provided to inspectors. The data shows staffing and occupancy on every day. It’s an unprecedented degree of granularity that allows for new levels of inquiry.
Low staffing is a prime factor behind many injuries among nursing home residents. As I wrote in the NYT article published earlier this month, “When nursing homes are short of staff, nurses and aides scramble to deliver meals, ferry bedbound residents to the bathroom and answer calls for pain medication. Essential medical tasks, such as repositioning a patient to avert bedsores, may be overlooked when workers are overburdened, sometimes leading to avoidable hospitalizations.”
Analyzing the files took time. To begin with, the files were too large to analyze in Excel — a challenge with more and more health care data sets. I previously had taken classes in SQL at the National Institute for Computer-Assisted Reporting, but this was the first project in which I had to use SQL. Fortunately, KHN has two great data journalists who steered, taught and backstopped me: data editor Elizabeth Lucas and data correspondent Sydney Lupkin.
There were many decisions to make during the analysis. Should we count nursing administrators in assessing staffing levels? How should we define best-staffed and worst-staffed days? How do we handle the fact that the data in its raw form wasn’t risk-adjusted — that is, that it didn’t tell you how comparatively frail and sick each nursing home’s residents were, making it dangerous to make comparisons. (You can read our decisions in our methodology, which accompanies the downloadable data.)
I guided myself with two intersecting principles: to reflect residents’ lived experience as accurately as possible, and to be fair to the facilities. When in doubt, I erred on the side of caution.
I bounced approaches off not only Liz and Sydney, but two top health care researchers who have published studies on staffing: David Stevenson at Vanderbilt University Medical Center and David Grabowski at Harvard Medical School. Both are straight shooters and rigorous in their methods. It was also invaluable to talk early in the project with representatives of the nursing home industry’s main trade group, the American Health Care Association, to get the group’s views about the strengths and weaknesses of the data set.
I’ve been writing about nursing homes for a while, but until this story I didn’t know the source value of SNF resident and family councils. CMS authorizes these councils, and facilities must let them meet on the premises and invite to meetings anyone they want, even journalists.
The leaders of the council at Beechtree Center for Rehabilitation & Nursing, the Ithaca, N.Y., home that provided anecdotes for my story, were a reporter’s dream: smart, dedicated and reasonable. They were able to see things from the facility administration’s position, but also were firm advocates for good care. They had educated themselves about the facility plus the rules and regulations. Adam Chandler, whose mom was a Beechtree resident, already had been studying the PBJ data. The council’s president, Stan Hugo, carried around a tan notebook in which he noted incidents of staffing problems he came across. He also kept a daily log of how many employees he observed when visiting his wife and how many were posted as being on duty on the sign Medicare requires facilities to post publicly.
Our story ultimately focused on the wide fluctuations in nursing home staffing from day to day, with staffing consistently plummeting on weekends. Caitlin Hillyard, one of KHN’s web producers, created a terrific interactive graphic that reporters can use to see staffing patterns in nursing homes in their state. We encourage other reporters to download and use the data for their own stories. There’s no cost, but please credit KHN and provide a link back to us.