Reporters who cover medical studies often take great care not to be fooled by the spin put on research by drug companies, universities and even government agencies.
But sometimes the spin is the study itself, and that’s a serious problem. It’s a big story that’s hiding in plain sight, to borrow a phrase Steven Brill likes to use.
By some estimates, half of clinical trials are unpublished. Half. And because positive studies are far more likely to be published than negative studies – a phenomenon called publication bias – the studies that don’t get published often throw some seriously cold water on how good a treatment looks.
If this research hasn’t been published, how do we know it exists? Some of these trials have been released because of lawsuits; others can be found in standardized documents called clinical study reports that drug companies file with the FDA and its counterpart, the European Medicines Agency. Regulatory agencies use them for their reviews, but because they’re never published in medical journals, they remain hidden to the medical community and general public.
For a case in point, consider the antidepressant reboxetine. Unpublished studies that were brought to light in a stunning 2010 meta-analysis in the British Medical Journal showed that Pfizer had failed to publish data – all of it negative – on 74 percent of patients who had participated in the clinical trials of the medication.
“Not only does the drug not work, it really doesn’t work,” wrote the blogger SciCurious in a guest post for Scientific American on the reboxetine revelations.
Hidden or unpublished clinical trial data also is at the heart of the recent questions about the effectiveness of the drug Tamiflu.
“This is a systematic flaw in the core of medicine,” said Dr. Ben Goldacre, a British physician and a leading scientific watchdog, in a 2012 TedMed talk. “This is a disaster. We cannot know the true effects of the medicines we prescribe if we do not have access to all the information.” Goldacre also is a driving force behind the AllTrials campaign, an Internet petition that aims to raise awareness about the issue.
Various remedies for the problem have been tried with limited success. Starting in 2008, the FDA Amendments Act required reporting of all clinical trials of drugs, devices, and biologics with at least one study site in the United States within a year of the study’s completion. But a 2012 study in the BMJ found that only 22 percent of the trials subject to the new regulation have complied with the law. A separate analysis, published in Pediatrics, found some companies were shutting down trials with negative results early in an apparent attempt to skirt reporting requirements. Proposed legislation clarifies that the law applies to trials regardless of their outcome. Well, duh. But thanks anyway, D.C.
Pharmaceutical companies have also repeatedly pledged to police themselves when it comes to reporting all their data.
Now some of the same researchers who have been working to release unpublished Tamiflu data think they may have found a fix. If studies are missing or misrepresented in the medical literature, they say outside authors should be able to publish or correct that research. In a paper and editorial published last week in the BMJ, the researchers dub this concept RIAT, for Restoring Invisible and Abandoned Trials.
In a sense, they’re calling for medical ghost writers, except these authors would get credit for their efforts and wouldn’t be paid by the trial sponsors. They’ve laid out a detailed plan for how outside authors can responsibly publish another researcher’s work.
RIAT could solve problems on several fronts. Young investigators, who are often under intense pressure to publish to further their careers, could find plenty of readymade data to dive into in these orphaned or ghost studies. That might help stem the rising tide of scientific fraud and could perhaps also put a dent in the raft of weak and unreliable data that’s picked up by predatory or junk journals.
“RIAT is about cleaning up the scientific evidence,” said Peter Doshi, Ph.D., a postdoctoral fellow in comparative effectiveness research at Johns Hopkins University School of Medicine in Baltimore, Md. “Really, I hope this can start to set a new standard,” he said.
Doshi and his co-authors say they have already identified more than 178,000 pages of unpublished data on common drugs such as Relenza, Seroquel and Plavix. In some cases, these are not small, inconsequential studies. Studies that remain unpublished include the largest trial ever conducted on Tamiflu and the largest study of the drug gabapentin, sold under the brand name Neurontin, for diabetic neuropathy. In other cases, studies that have been published appear to be incomplete because they don’t include missing data.
So far, three medical journals, the BMJ, PLoS One and PLoS Medicine, have signed on to the idea and agreed to consider RIAT manuscripts, provided that authors first follow certain steps designed to give the original researchers a chance to set the record straight themselves.
The initiative is important, the journal editors write, because “public confidence in the credibility of medical research is at a low ebb.”
So keep an eye out for RIAT studies. Let’s hope enterprising authors embrace the concept and take advantage of the offers made by these journals to publish the work.
Because when we’re only getting seeing half of the studies, we’re only getting half the story.
