By Tara Haelle
You run across (or are sent) a fascinating study that seems newsworthy – but it’s published in a medical journal you’ve never heard of. How do you make sure it’s a legitimate, reasonably high-quality publication?
Before we answer that, let’s consider why that question is important in the first place. In every field of science, a loose hierarchy of journals exists in terms of prestige. In medicine, for example, the New England Journal of Medicine, Science, Nature, the Lancet, JAMA (and its affiliated journals) and BMJ tend to be regarded most highly.
These journals have been around a long time and are fairly reliable and consistent in the quality of the research they publish. They have a clearly established peer review process, and their articles are frequently cited by other papers (more on that in a minute). There also are medical specialty-specific journals, such as Heart or Pediatrics or Neurology, which tend to be at the top of the stack within their respective fields.
But often some of the most interesting findings can come from a smaller journal, especially in an emerging area of science that isn’t widely studied or accepted, yet remains scientifically sound. Make sure that the journal is not part of the growing trend of so-called predatory journals — that is, exploitative open-access journals that charge scientists to publish their work without providing adequate editorial oversight. That includes peer-review and vetting of the article’s significance and the quality or accuracy of the science involved. Literally anyone who can pay can publish anything in a predatory journal.
In other instances, a journal may not be predatory, but it may simply be of extremely low quality for one reason or another. It also might be affiliated with an advocacy organization, which can compromise editorial independence.
So, how do you vet a journal before moving forward on reporting a study published in it? Here are some guidelines on performing due diligence on the journal’s quality:
Pay to publish?
Investigate whether it’s a predatory journal. This is not easy task since predatory journals are constantly popping up, but you can start by seeing whether the publisher is on this list of predatory publishers. If it is, stop right there — the reliability of the science is potentially too questionable to report on. If the publisher isn’t listed there, check whether the journal itself is listed among Beall’s list of predatory journals. Though neither comprehensive nor perfect, this list is frequently updated.
New guy on the block?
If you can’t find evidence of the journal being predatory on this list or through online searches, look at how long it’s been around. When was the first article published? If it’s only been in the past few years, that might be a red flag, but not definitely. There are a variety of new journals popping up all the time with new publishing models online, and being the new kid on the block is not necessarily bad in and of itself. It does mean you’ll want to dig a bit further.
Does PubMed care?
Is it indexed in PubMed? This could actually be listed as the first step, but being listed in PubMed is not, in and of itself, a guarantee that it’s a reputable journal. The odds are higher but certain. Further, it’s just a pain to wait for the list of tens of thousands of journals to load so you can do a keyword search.
Who is citing it?
What’s the impact factor? Here’s one place to look. This number represents how frequently research papers in that journal are cited by other papers. The higher the number, the more that journal’s research is cited, which generally tends to correlate with a higher quality and more reliable journal. While not a perfect measure of quality or individual studies published, this at least provides some sense of an objective measure. If the publication doesn’t have an impact factor, that could be a red flag unless it’s less than two years old (in which case it may not have been assessed yet).
Author’s track record
Where else have the authors been published? If you’ve been through all the previous steps and you’re still uncertain about the journal — it’s in some kind of middle ground — use PubMed to look up each of the authors, starting with the lead author (the first name on the author list) and the senior author (the last name on the list).
In theory, these two individuals should be most involved in the paper’s research, and in most cases, they have probably published something before. The lead author may not have this if they are a graduate student publishing their first piece, but in that case the senior author will almost certainly be their adviser and should have published plenty. If the authors have been published in other reputable and well-established journals, chances are higher that this is a reliable paper. There is still a question to explore about why the paper is in a less well-known journal, but that’s a question you can ask the researchers during your interview.
On the other hand, if virtually none of the authors have been published before, or if they have been published in other questionable journals, it’s another red flag.
Ask around
If you are still leaning toward reporting on the study, but still want more information about the quality of the journal, ask other doctors, scientists and researchers in the same field. Anyone who publishes in that field will know the hierarchy of journals for their area of research and should be able to tell you about how highly a journal is considered among peers. If they’ve never heard of the journal, that could be a red flag. Even if the journal has an impact factor and is indexed in PubMed, you might be told that it’s still a pretty poorly regarded journal and you should tread carefully.
These tips aren’t foolproof ways to guarantee that study is a good or bad, but the quality of the journal can go a long way in accessing how accepted the research is likely to be in the field.
As AHCJ’s medical studies core topic leader, Tara Haelle (@tarahaelle) guides journalists through the jargon-filled shorthand of science and research and enable them to translate the evidence into accurate information that their readers can grasp.
