By Brenda Goodman
Recently, Dr. Ben Goldacre (@bengoldacre), a prominent critic of drug studies, embarked on a research project of his own. He wanted to find out how often side effects reported by users of cholesterol-lowering drugs called statins were genuinely caused by the medications.
The study he co-authored concluded that most reported side effects of statins aren’t often due to the drugs themselves, but to other causes – perhaps an unhealthy lifestyle that contributes to heart disease in the first place, or even the nocebo effect, which occurs when people experience side effects because they expect a drug will cause them.
The study generated front-page headlines in the U.K., with an article in The Telegraph declaring, “Statins have virtually no side effects, study finds.”
Outcry ensued. Patients who experienced side effects on statins begged to differ, and Goldacre’s fans wondered if he had suddenly gone soft on pharmaceutical companies.
In response, Goldacre penned a nuanced explanation of the study findings, admitting that its conclusions were flawed because it was based on incomplete data.
The statin study controversy aside, his blog post makes some key points about the way side effects are reported in medical journals. These are helpful for health reporters to keep in mind when covering the downsides of new drugs:
- Most side effects aren’t reported in journal articles. How do we know that? German scientists recently compared the side effects reported in Clinical Study Reports (CSRs) – the lengthy and detailed trial reports that companies are required to submit to regulatory agencies like the FDA – to the side effects reported in journal articles on the same trials. From Goldacre’s post:
“…a recent study by IQWiG, the German government’s cost effectiveness agency, found complete information for 87% of adverse event outcomes in the standard lengthy regulatory document for industry trials (the Clinical Study Report) but for only 26% of adverse event outcomes in the journal publication (Wieseler, 2014).”
- Drug companies aren’t very motivated to look for side effects. From the post:
One pointer towards this is that, although liver transaminase elevation was documented in the majority of trials, new diagnosis of diabetes was only documented in three of the 29 trials.
Liver transaminase is an enzyme that can be a marker for liver damage, but frequently, it can go up without any real harm. Meanwhile, other studies have suggested that statin use may lead to type 2 diabetes, so it seems like that would have been important to look for.
- Many studies exclude patients with chronic conditions other than the one that’s under study. They may also screen out patients taking drugs that might share the same metabolic pathways in the body. But in the real world, people often have more than one thing going wrong at the same time and they often take more than one medication. That could make them more vulnerable to side effects than the ideal patients included in the trial.
- Clinical trials don’t usually state how and how often side information on adverse effects is collected. (So ask about this in your interview!)
- Look for something called a run-in phase. Run-in phases are a period before the trial starts when all possible study participants are given the placebo, or the medication, or sometimes both, to screen out certain patients. Sometimes run-in phases sharpen a study’s findings in important ways, but they can also skew the results. For example, patients might be asked to take a placebo to screen out patients who are irregular pill takers. This might result in a study of highly motivated patients, who aren’t necessarily representative of the real world. Other run-in phases are meant to screen out patients who report side effects early on, perhaps decreasing the true population of people who experience side effects on the drug. Ask about the purpose of the run-in phase and find out whether the results from the run-in were included in the final analysis.
Brenda Goodman, AHCJ’s topic leader on medical studies, is writing blog posts, editing tip sheets and articles and gathering resources to help our members cover medical research. If you have questions or suggestions for future resources on the topic, please send them to brenda@healthjournalism.org.
