Pfizer made waves Monday with its announcement that its COVID-19 vaccine, developed with partner BioNTech, is “strongly effective,” with a reported efficacy of over 90%. The news was so highly touted that I woke up to multiple texts from friends about it, and it definitely sounds exciting.
The problem? That 90% is almost the only number we know because the company didn’t release additional data for others to read and interpret. Once again, these “extraordinary” findings, as Pfizer’s senior vice president described them to Stat News, were shared as “data by press release,” a worrisome trend during a pandemic.
The only other meaningful number available is that the phase 3 trial, which had enrolled 43,538 participants, included 94 confirmed COVID-19 cases. Phase 3 trials are the final phase before submission to the FDA for approval, and their primary purpose is to determine efficacy while watching for any potentially rare adverse effects. They cannot conclude until there are enough disease cases among participants for the researchers to compare vaccinated and unvaccinated groups.
To achieve at least a 90% efficacy in the interim analysis, no more than nine of those cases could have been people who received the vaccine, with the others in the placebo arm. We don’t know yet how many cases were in each group because Pfizer didn’t release that information.
What’s important to keep in mind is that this is an interim analysis. That means the study did not have enough cases, as determined before the study began, to draw any conclusions about the primary endpoint, which is the vaccine’s efficacy in terms of COVID-19 prevention. Interim analyses frequently are presented at conferences, so it is common to release them, but they rarely get coverage as wide as Pfizer’s news. Readers need to understand that 90% does not represent the final efficacy determined by the end of the trial. An interim analysis tells us which way the winds are blowing, but there’s still time to get blown off course.
Matthew Herper’s analysis for Stat News does a great job of laying out what we know and what it might mean. But the bottom line is that even though this news provides an encouraging bit of optimism, we still don’t know much. It’s admittedly hard not to hope. A vaccine will be key to getting life one step closer to some semblance of normalcy, and various experts on Twitter did express cautious optimism.
It’s important to keep asking questions, wait for all the data, avoid hype, and help readers and audiences understand what this news does and does not mean.
AHCJ infectious disease core topic leader Bara Vaida and I recently wrote a tip sheet and blog for reporting on COVID-19 vaccines. I’ve previously written about what to look for in COVID-19 vaccine trials, red flags to watch for in the trials, and the different ways to assess efficacy in vaccine trials. If you’ve read any of those, you realize just how much information is missing from Pfizer’s announcement and why the need for skepticism and patience is so essential.
For more on what questions experts are asking — the same questions journalists need to be asking — see this thread from epidemiologist Ellie Murphy and this thread from virologist Angie Rasmussen.
