I’ve written in previous posts about what to look for in COVID-19 vaccine trials and red flags to monitor. The two most important outcomes in vaccine trials are the vaccine’s safety and its efficacy. Recall that efficacy is different from effectiveness: efficacy refers to how well the vaccine prevents infection in the clinical trial, with effectiveness referring to how well it prevents infection in the real world with a broader and more diverse population.
But it’s important to understand how vaccine efficacy is calculated, and other aspects of efficacy that can ensure your reporting on vaccine trial results is precise and accurate. Vaccine expert Natalie Dean, Ph.D., an assistant professor of biostatistics at the University of Florida, has posted a helpful tweet thread laying out all you need to know.
You’ll want to bookmark the second tweet in the thread or save the photo because it shows the relationships between efficacy and relative risk, risk ratio, incidence rate ratio and hazard ratio, any one of which may be reported in the study.
The image is a key for knowing what language to use, no matter how the data are reported: take the risk of infection in the vaccinated arm divided by the risk of infection in the unvaccinated arm, and subtract that quotient from 1 to get vaccine efficacy.
“Vaccine efficacy (VE) measures the relative reduction in infection/disease for the vaccinated arm versus the unvaccinated arm,” Dean wrote. “A perfect vaccine would eliminate risk entirely, so VE = 1 or 100%. This can be calculated from the risk ratio, incidence rate ratio, or hazard ratio.”
Then she explains what it means when that number is less than 100%. Vaccine efficacy of 50% means that a vaccinated person is half as likely to get sick as someone unvaccinated — or someone unvaccinated is twice as likely to get sick as someone vaccinated. More precisely, she points out that a 50% chance refers to the likelihood of infection when both people are exposed to enough virus for infection. However, it’s usually not necessary to get that granular.
But here’s where it gets more complicated: Efficacy isn’t just about preventing infection. That’s just one type of efficacy, one that offers sterilizing immunity — complete immunity such that no infection takes place at all.
There’s also efficacy to prevent disease, which means you might become infected — the virus might replicate in you — but you never develop symptoms or the disease. (Evidence suggests this can happen with the acellular pertussis vaccine, for example, where the bacteria can still colonize in a vaccinated person — potentially being contagious — but the person never develops the disease.)
And there’s efficacy to prevent severe disease. Though Dean doesn’t mention it here — it may be premature to measure it — but you can also consider the efficacy or effectiveness of the vaccine in preventing hospitalizations or preventing deaths, both of which are often left out when people talk about the seasonal flu vaccine’s frequently low effectiveness. (Even when its effectiveness is low in preventing infection, it’s often better at preventing hospitalization and death.)
Dean points out that the primary endpoint in phase 3 trials usually is efficacy in preventing disease. Efficacy against both infection and severe disease usually are secondary endpoints. But you should read the study carefully to be sure you understand what efficacy the researchers are reporting — and whether that primary outcome was defined before initiating the study. As Rajeev Venkayya, M.D., also points out on Twitter, many vaccines actually are more successful at preventing severe disease than mild illness. Again, the influenza vaccine is a good example.
Then there’s population-level protection, which is how well a vaccine contributes to herd immunity by reducing infectiousness to others who aren’t vaccinated.
“A vaccine that prevents infection entirely provides indirect protection to others,” Dean wrote. “If I can’t get infected, I can’t infect you. But it is possible to have a vaccine that prevents disease, but individuals can still be infectious.”
An example of the latter is again the acellular pertussis vaccine, as well as influenza vaccines, as pointed out by Columbia vaccinologist Florian Krammer. (By the way, that thread by Krammer is worth a blog post of its own and the most incredible tweetorial I’ve seen on Twitter.)
How do researchers assess indirect protection? One way is looking at household transmission, Dean said.
The bottom line is that “vaccine efficacy” encompasses many different definitions and numbers. It is important that health journalists covering phase 3 COVID-19 vaccine trials understand the differences and communicate those differences to readers, viewers and listeners.
