#AHCJ17 panel explores ongoing quest for reliable measures of hospital quality

Health reporters should be asking the hospitals they cover plenty of in-depth questions about their star ratings and other collected quality measures. But they should not assume that those measures reflect the hospital’s true performance.

That was the takeaway from an engaging panel on hospital quality measures held at AHCJ’s Health Journalism 2017 conference in Orlando.

“It is absolutely astonishing how much effort, and frankly, waste is expended to measuring hospital quality,” said Don Goldmann, M.D., chief medical and scientific officer at the Institute for Healthcare Improvement, a quality improvement group in Boston.

Not only is the onus too great on hospitals and providers to collect and report quality data, panelists said, but often those tasks are not part of standard work, taking valuable time away from patient care. They noted that on average it costs $40,000 per physician each year to collect and report required quality metrics.

Much of today’s quality reporting is driven not only by a push for greater transparency in care but also by pay-for-performance (or value-based) payment changes, in which providers must show they are improving care to receive a share of reimbursements, panelists said.

The flawed design of many quality measures also means that the results do not always reflect quality care or improving care. Also, much of the data needed is not extractable from electronic health records, so it requires other means of gathering.

“For instance, there’s lots of great data about specialists out there, but it is not linked to the EHR or accessible to patients,” Goldmann said.

Robert Cima, M.D., professor of surgery at the Mayo Clinic College of Medicine, used surgical site infection (SSIs) problems as an example of how reporting quality metrics can be a challenge.

Gathering data on SSI is often person-dependent, and therefore not reliable due to billing coding errors, misunderstandings or lack of guidance about which conditions should be recorded as a SSI, Cima said

One hospital had a different SSI infection rate depending on how the data was gathered and recorded, he said. In a manual chart review (perhaps the most accurate way to measure data, but also time consuming), the hospital SSI rate was three times the rate than when the data was collected in other ways.

“You can’t play baseball with three different sets of rules,” Cima said.

Beyond data collection measures, reporters should probe what exactly is being measured, advised Kevin J. Bozic, M.D., professor and chairman of the Dell Medical School Department of Surgery and Perioperative Care at the University of Texas, Austin.

Process measures (showing evidence-based practices are followed); structural measures (such as electronic health record adoption tracking); and experience measures (such as patient satisfaction survey results) don’t necessarily correlate with patient outcomes, Bozic said. Measuring costs does not always mean better quality either.

The most important way to measure quality is through outcome data, such as how well a patient did after surgery or whether a diabetic’s blood sugar is under control over time. “Outcomes measured from the patient’s perspective are the most important measures,” Bozic said. “Improving quality of life is most important to patients.”

Panelists suggested these questions for reporters to ask about quality measures:

  • How is the data gathered by hospitals?
  • Is that data being gathered in the same way as the hospital’s competitors?
  • Is what is being measured relevant for patient health outcomes?
  • Is the hospital participating in quality reporting efforts (and if not, why not?)?
  • Does the hospital have other data sources to share, such as patient disease registries, which can be rich data sources?

Signs of a good quality hospital include a willingness to publish quality measures, regardless of high or low scores, and consistent scores across several different quality reporting programs.

Journalists seeking data on specialty care should look to specialty medical societies, which often run their own quality reporting programs but are not well-publicized, panelists said.

“We need to do a better job of presenting data to you, the journalists, to explain them to the public in a way that patients can understand,” Goldmann said.

See the tweets from this panel for more tips.


Health Journalism 2017

1 thought on “#AHCJ17 panel explores ongoing quest for reliable measures of hospital quality

  1. Avatar photoMichael Millenson

    As someone who has followed the quality field for many years, writing for both newspapers and a book, I am disappointed that the fundamental reluctance of hospitals to report anything to the public was not in this story. The question journalists should be asking is: “What numbers do you use internally? I, as a reporter, will have to wait months for the claims-based data from Medicare to become public. Since you are a non-profit/academic hospital whose mission is to serve the public, I’d like you to share your numbers with me now. Will you do that?”

    In other words, don’t get caught up in the weeds and allow that discussion to get hospitals off the hook (to mix metaphors). It is, after journalists who have been behind the push for transparency, as my piece in the medical literature (“The Press Pushing the Profession”) shows. Available free here: http://qualitysafety.bmj.com/content/11/1/57

Leave a Reply