Graphic by Tara Haelle
The number of promotional adjectives used to hype proposed research has increased by more than 1300% in successful grant applications to the National Institutes of Health over the past three and a half decades, particularly in hyping the importance and novelty of research, according to a recent study in JAMA Network Open.
The use of “spin,” the authors wrote,” has the potential to undermine the fidelity of scientific reporting,” but the identification of spin in the peer review process is problematic, they add.
”In part, the challenge for authors, reviewers and readers may be a paucity of information on how spin is realized at the sentence level,” they wrote. “Thus, we are advised to look out for spin, but not told precisely what the language of spin looks like.” Spin refers to “specific intentional or unintentional reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression the results produce in readers,” according to the study.
Journalists may be a bit more attuned to looking for this language than other more casual readers, but the hype used in scientific language can be more subtle than in other places. And while journalists don’t usually read grant applications, the same diction used in the abstracts of grant proposals to the NIH also appears in the abstracts and main text of studies published in peer-reviewed journals. Though grant applications are a bit more explicitly marketing-oriented as researchers seek funding for their projects, published studies — despite their intended purpose of describing an experiment and its findings — are also a kind of currency since publications are one measure used to award tenure and other benefits, including grants to researchers.
Hence health journalists should look out for similar language in studies they read and cover. Consider the following takeaways from this study:
- Be aware of what words in scientific research can serve the promotional purpose of hyping the research.
- Guard yourself against falling into the trap of thinking that those words are necessarily accurate, necessary or appropriate descriptions of the research when covering the study.
- Consider not quoting any statements from the study that include those words.
- When assessing the quality and value of a study for public consumption, focus on the actual data more than the authors’ interpretations of the data.
- Always seek commentary from outside sources unrelated to the research as a check on potential hyping of the findings.
- Consider why a particular study was funded and whether hype language may have played a role in the researchers getting funding for it.
To determine whether the use of “hype” language in grant applications to the NIH had changed over time, the researchers examined the use of promotional adjectives in funded NIH grant application abstracts from 1985-2020, more than 900,000 abstracts. They identified 139 different adjectives as the hype. Nearly all of these increased by 7,690 words per million over the time period studied — a nearly 1400% increase. Only nine hype adjectives decreased in frequency and did so by 44%.
The adjectives used are not necessarily the over-the-top exaggerations that might be found in irresponsible news stories, such as “promising” or “dramatic” or “miracle.” They are words that in a scientific context, can still imply great promise, importance or relevance even if they are dryer than what’s seen in more casual settings.
Accordingly, the words whose usage increased the most in absolute terms were “novel,” “critical” and “key.” Novel increased by 1,054 words per million while critical and “key” increased by about half that. (If you’re someone who reads or covers a lot of studies, you know how common these terms are in published research.)
The words whose usage increased the most in relative terms are also words that likely weren’t as common in general parlance in the 1980s: sustainable increased by more than 25,000%, actionable increased by over 16,000%, and scalable increased by just over 13,000%.
Other value-laden adjectives that can be used to hype research include “important,” “original,” “robust,” “innovative” and “unprecedented” — all words that a previous study showed have increased in PubMed abstracts. Others identified in this study included “crucial,” “powerful,” “accurate,” “excellent,” “transformative,” “groundbreaking” and “stellar.” The authors grouped the adjectives into eight broad categories listed here with some examples:
- Importance: key, paramount, imperative
- Novelty: innovative, unique, unprecedented
- Rigor: careful, rigorous, scientific
- Scale: enormous, myriad, vast
- Utility: effective, impactful, easy
- Quality: qualified, skilled, talented [researchers] or premier, renowned, vibrant
- Attitude: attractive, exciting, incredible
- Problem: alarming, dire, elusive
The authors acknowledge that the persuasive nature of grant applications makes it unsurprising to find words like these in them but say that the overall trend is still problematic. Hype might bias the evaluation of research and can undermine clarity. Some of these terms may be appropriate in context, while others are gratuitous or buzzwords that journalists should be cautious not to echo.