As various COVID-19 vaccine candidates make their way through clinical trials — see this nice update on where things stand from Helen Branswell at STAT — journalists need to be scrutinizing the findings as closely as possible when reporting on them. But what do you look for?

The questions I include from this piece from Elemental, primarily aimed at laypersons, are a good starting point. Then, getting more detailed, look to this brief thread of tweets from Vinay Prasad, M.D., a hematologist-oncologist and associate professor of medicine at the University of California, San Francisco.

Below are the key elements he recommends looking for in a vaccine trial, with additional commentary or explanations on the AHCJ website:

• • Blinding: A vaccine clinical trial should be double-blinded so that neither the participants nor study researchers administering the vaccine know which ones are the actual vaccine candidate and which are a placebo.
  • Randomization: Participants should be randomly assigned to the groups that receive either the real vaccine or the placebo. There should be a comparable breakdown of demographics in both groups. Look for whether there are statistically significant differences between the make-up of the vaccine group and the placebo group.
  • A pre-specified primary endpoint and power calculation. The trial should have a primary endpoint, stated before the trial begins (check ClinicalTrials.gov for this), that determines what will be the most important outcome measure from the trial. In a vaccine trial, it could be an antibody response (titers), the percentage of people who get infected, or another measure of immune response. A power calculation refers to how many patients are needed in the study to produce a reasonable effect size with statistical significance. This likely isn’t something a journalist will be able to assess on their own (unless they have a strong statistics background). They can still use this Sample Size Calculator for a rough estimate, and a biostatistician can provide you with context and analysis.
  • “That endpoint should be a meaningful, consistent, patient-centered endpoint of infection or illness,” Prasad wrote. This is precisely the kind of question you can ask researchers when reporting on vaccine trials.
  • “The trial should be powered to detect a meaningful effect size.”
  • “Safety signals should be examined carefully,” Prasad wrote. “Very carefully.” It’s worth remembering the difference between adverse events and side effects here.