Tag Archives: surgeons

ProPublica offers rebuttal to Rand critique of Surgeon Scorecard

Pia Christensen

About Pia Christensen

Pia Christensen (@AHCJ_Pia) is the managing editor/online services for AHCJ. She manages the content and development of healthjournalism.org, coordinates AHCJ’s social media efforts and edits and manages production of association guides, programs and newsletters.

star-ratingNot unexpectedly, ProPublica has published a response to the Rand Corporation’s recent 20-page critique of the journalism organization’s Surgeon Scorecard, a searchable database of complication rates for surgeons performing several elective operations.

In a Rand summary of the piece, its authors advised patients “not to consider the Scorecard a valid or reliable predictor of the health outcomes any individual surgeon is likely to provide.” The paper listed what researchers labeled “Methodological Issues” in the Scorecard and ways the authors thought the data could be improved.

ProPublica’s rebuttal, written by editor-in-chief Stephen Engelberg  and deputy data editor Olga Pierce, offers specific responses to Rand’s points and says the organization plans to take some suggestions into account as it prepares the next version of the scorecard. However, it says conversation about the database should be “grounded in fact and evidence.” Continue reading

Rand analysts point out limitations in ProPublica’s Surgeon Scorecard

Joseph Burns

About Joseph Burns

Joseph Burns (@jburns18), a Massachusetts-based independent journalist, is AHCJ’s topic leader on health insurance. He welcomes questions and suggestions on insurance resources and tip sheets at joseph@healthjournalism.org.

It’s inevitable that when any organization publishes a report card on physician quality, doctors and policy experts will critique its methodology. Therefore, ProPublica surely is not surprised see criticism of the extensive work that Sisi Wei, Olga Pierce and Marshall Allen did sorting through Medicare billing records for 17,000 U.S. surgeons at 3,575 hospitals who performed eight types of surgery from 2009 through 2013. The analysis formed the database of the nonprofit newsroom’s Surgeon Scorecard launched in July. (We covered the scorecard here: Surgeons’ complication rates become public with new database.)

The latest criticism comes from the Rand Corp. In a 19-page report issued today, A Methodological Critique of the ProPublica Surgeon Scorecard, nine Rand researchers say their critique had three goals: to explain issues they have with the scorecard’s methodology; to suggest ways to improve the scorecard; and to inform the public about these issues. By informing the public, the researchers said in their Sept. 25 critique, “…we hope patients who are choosing a surgeon will be better able to decide how much weight to give the data presented in the scorecard.”

In producing an online, searchable database for the public, ProPublica’s goals are laudable, the researchers said, but the scorecard’s ability to achieve its goals is limited by the rigor of the methods and the adequacy of the underlying data. They point out five specific issues they have with the methodology, and outline six ways to improve the scorecard. Continue reading

Surgeons’ complication rates become public with new database

Pia Christensen

About Pia Christensen

Pia Christensen (@AHCJ_Pia) is the managing editor/online services for AHCJ. She manages the content and development of healthjournalism.org, coordinates AHCJ’s social media efforts and edits and manages production of association guides, programs and newsletters.

Whether consumers are choosing a car, a household appliance or even a nursing home, there are ratings and reviews available to make the best choice. But patients are often blind when choosing a surgeon.

Surgeon Scorecard, a database released by ProPublica this week helps shed some light on that area with an analysis of death and complication rates for nearly 17,000 U.S. surgeons for eight common surgical procedures. This is the first time this information has been available to the public. Continue reading