Spotting Spin: A Brief Guide to Honest Scientific Reporting

First posted on the ORIGINal Thoughts Blog

Randy Townsend, MPS
Consultant, Origin Editorial
LinkedIn: https://www.linkedin.com/in/randy-townsend-mps/

Take Home Points:

• Reporting bias
• Inappropriate interpretation
• Inappropriate extrapolation of results
• Inappropriate attribution of causality

A research team excited by a new discovery may focus their reporting on data that contributed to a primary outcome, with little or no mention of null or negative results, or secondary outcomes with significant results. Reporting bias occurs when the decision on what to report is directed by the impact of the findings. This type of reporting creates an imbalance that may obscure the full picture and compromise the integrity of public trust. This is particularly concerning when financial implications, political leverage, or corporate agendas with vested interests influence research outcomes.

A study published in BMC Medicine reported that news articles that included spin in their coverage of a study about a particular treatment left readers with a positive impression of the treatment’s benefit. The authors noted that “frequent misleading reporting included stories emphasizing benefits, a focus on relative instead of absolute risk reduction; relying on anecdotes; and omitting harms and study limitations.” The authors went on to say that the spin in the news stories was related to “the misrepresentation of study results in the research report,” meaning the spin originated in the research phase and extended to widespread dissemination.

In legal terms, leading questions (definition from the Fourth Judicial Circuit Court of Illinois) suggest the answer. In other words, the lawyer leads the witness to say what the lawyer wants the witness to say. With spin, oftentimes there are subtle traces of influence that can activate the power of suggestion, intentionally directing our attention towards a desirable conclusion. In a study published in the Journal of Clinical Oncology,  authors evaluated the impact of spin on the interpretation of results of abstracts of randomized controlled trials in the field of cancer research. Their results demonstrated the influence of spin, noting that it “may be responsible for a gap between the perceived and actual beneficial effect of the treatments studied.”

The subtleties of spin aren’t limited to medical publishing. Spin can also be deliberately weaponized in geopolitical debates and territorial disputes. The 2023 blockbuster “Barbie” movie briefly showed a map that featured China’s Nine-Dash Line. This imaginary dashed line sometimes appears on maps of the South China Sea, loosely implying China’s territorial claim to the disputed region, which includes Malaysia, the Philippines, and Vietnam. According to CNN, “a 2016 ruling by an arbitration tribunal in The Hague invalidated China’s claim to almost the entire waterway where trillions of dollars in ship-born trade passes” each year. Geoscientists, who “include geologists, paleontologists, seismologists, meteorologists, volcanologists, hydrologists and oceanographers,” often use maps to define the specific region(s) examined in their research. While the Nine-Dash line isn’t traditionally top of mind for peer reviewers focused on evaluating geoscience research, there are times when it is present in author submissions. There are concerns that, when published, it could be used as propaganda to support and justify a particular position and inflame ongoing international tensions.

Nguyen Thuy Anh, a Vietnamese scholar and research fellow at the East Sea Institute of the Diplomatic Academy of Vietnam, identified journal articles as the latest form of propaganda used by China to legitimize their claim to the region. He discovered nearly 300 published articles using the Nine-Dash Line in prominent scientific journals.

Barbie is not the only film that raised eyebrows because of the inclusion of the Nine-Dash Line. DreamWorks Animation’s 2019 animated file, Abominable, the 2021 Netflix series “Pine Gap,” and the 2022 Sony Pictures action movie, “Uncharted,”  all displayed maps that featured the controversial lines and were the subject to regional protests and national bans.

OpenStreetMap Foundation (OSMF)

For Authors: Words of Caution

Research findings should attempt to remain as objective as possible. Pressure to produce high-impact results, similar to Sasai and Obokata, could influence a decision to include slanted language in the research intended to create and legitimize a particular narrative. Research that is overly positive may lead readers to overlook negative results. Misleading, vague, emotional, or overinflated words could have implications that paint a certain kind of picture. Here are some words that should be used with caution, if used at all.

Please note that this does not automatically disqualify the use of these words in research reporting. If these, or similar words, make sense, there should be sufficient data presented to justify the use. They are only semantically sound if they are equitably evidenced-based.

What Should Editors and Reviewers Look For?

It’s not always easy to identify spin. Carefully placed words, slight exaggerations, and minor adjustments to a calculation can be strategically camouflaged, especially if the spin is intentional. The impact of such deceptive subtleties may not scream “fraud,” especially when data replication falls within the margins of error.

• Scrutinize studies that amplify positive results while minimizing or omitting negative or non-significant findings.

• Identify ambiguous language that is detail deficient.

• Be mindful of your own biases, whether institutional, geographic, political.

• Flag overly positive or inflated language that transforms the tone of the research from objective to advertisement. Ask clarifying questions when the appearance of imbalance arises.

• Interrogate potential conflicts of interest, including institutional affiliation, employment, financial relationships (including funding sources), or other kinds of influence that could compromise objectivity and research integrity.
• Verify that claims are adequately supported by reproducible data.

It is a shared responsibility of authors, editors and reviewers to protect the integrity of research reporting from spin. With many spheres of influence weighing on researchers, a peer review that deliberately adopts a lens for identifying spin helps to ensure the broader public receives objective research reports in a transparent manner that allows for unfettered, informed decision-making.