Not unexpectedly, ProPublica has published a response to the Rand Corporation’s recent 20-page critique of the journalism organization’s Surgeon Scorecard, a searchable database of complication rates for surgeons performing several elective operations.
In a Rand summary of the piece, its authors advised patients “not to consider the Scorecard a valid or reliable predictor of the health outcomes any individual surgeon is likely to provide.” The paper listed what researchers labeled “Methodological Issues” in the Scorecard and ways the authors thought the data could be improved.
ProPublica’s rebuttal, written by editor-in-chief Stephen Engelberg and deputy data editor Olga Pierce, offers specific responses to Rand’s points and says the organization plans to take some suggestions into account as it prepares the next version of the scorecard. However, it says conversation about the database should be “grounded in fact and evidence.” Continue reading
It’s inevitable that when any organization publishes a report card on physician quality, doctors and policy experts will critique its methodology. Therefore, ProPublica surely is not surprised see criticism of the extensive work that Sisi Wei, Olga Pierce and Marshall Allen did sorting through Medicare billing records for 17,000 U.S. surgeons at 3,575 hospitals who performed eight types of surgery from 2009 through 2013. The analysis formed the database of the nonprofit newsroom’s Surgeon Scorecard launched in July. (We covered the scorecard here: Surgeons’ complication rates become public with new database.)
The latest criticism comes from the Rand Corp. In a 19-page report issued today, A Methodological Critique of the ProPublica Surgeon Scorecard, nine Rand researchers say their critique had three goals: to explain issues they have with the scorecard’s methodology; to suggest ways to improve the scorecard; and to inform the public about these issues. By informing the public, the researchers said in their Sept. 25 critique, “…we hope patients who are choosing a surgeon will be better able to decide how much weight to give the data presented in the scorecard.”
In producing an online, searchable database for the public, ProPublica’s goals are laudable, the researchers said, but the scorecard’s ability to achieve its goals is limited by the rigor of the methods and the adequacy of the underlying data. They point out five specific issues they have with the methodology, and outline six ways to improve the scorecard. Continue reading
Whether consumers are choosing a car, a household appliance or even a nursing home, there are ratings and reviews available to make the best choice. But patients are often blind when choosing a surgeon.
Surgeon Scorecard, a database released by ProPublica this week helps shed some light on that area with an analysis of death and complication rates for nearly 17,000 U.S. surgeons for eight common surgical procedures. This is the first time this information has been available to the public. Continue reading