Tag Archives: rand

Rand analysts point out limitations in ProPublica’s Surgeon Scorecard

Joseph Burns

About Joseph Burns

Joseph Burns (@jburns18), a Massachusetts-based independent journalist, is AHCJ’s topic leader on health insurance. He welcomes questions and suggestions on insurance resources and tip sheets at joseph@healthjournalism.org.

It’s inevitable that when any organization publishes a report card on physician quality, doctors and policy experts will critique its methodology. Therefore, ProPublica surely is not surprised see criticism of the extensive work that Sisi Wei, Olga Pierce and Marshall Allen did sorting through Medicare billing records for 17,000 U.S. surgeons at 3,575 hospitals who performed eight types of surgery from 2009 through 2013. The analysis formed the database of the nonprofit newsroom’s Surgeon Scorecard launched in July. (We covered the scorecard here: Surgeons’ complication rates become public with new database.)

The latest criticism comes from the Rand Corp. In a 19-page report issued today, A Methodological Critique of the ProPublica Surgeon Scorecard, nine Rand researchers say their critique had three goals: to explain issues they have with the scorecard’s methodology; to suggest ways to improve the scorecard; and to inform the public about these issues. By informing the public, the researchers said in their Sept. 25 critique, “…we hope patients who are choosing a surgeon will be better able to decide how much weight to give the data presented in the scorecard.”

In producing an online, searchable database for the public, ProPublica’s goals are laudable, the researchers said, but the scorecard’s ability to achieve its goals is limited by the rigor of the methods and the adequacy of the underlying data. They point out five specific issues they have with the methodology, and outline six ways to improve the scorecard. Continue reading

New online tool evaluates health reform scenarios

Pia Christensen

About Pia Christensen

Pia Christensen (@AHCJ_Pia) is the managing editor/online services for AHCJ. She manages the content and development of healthjournalism.org, coordinates AHCJ's social media efforts and edits and manages production of association guides, programs and newsletters.

The RAND Corporation has released an online tool designed to help people “understand, design, and evaluate health policies.” The tool, COMPARE (Comprehensive Assessment of Reform Efforts), has sections about current health care, policy options, an analysis of options and modeling estimates. A “Policy Options Dashboard” allows users to explore the effects of commonly proposed health care reforms by customizing the options and clicking on links that explain concepts.

According to Healthcare IT News,

“Results on the dashboard are informed by a review of prior experience with the policy options and a microsimulation modeling tool that allows policymakers to estimate the impact of specific policy changes on coverage, spending, consumer financial risk and health.”

The “Modeling Estimates” section lets users select parameters and scenarios to see the effect such choices would have on the health care system.

The “Proposals” area of the site offers information about state initiatives in health care policy that have been porposed or implemented, as well as options proposed in federal legislation and proposals made by a number of professional organizations, private foundations and other groups.

The site includes a detailed explanation of what it is, how it works and who the funders and advisory committee are.