Double-Anonymous Peer Review: Why it Reduces Bias and Increases Equity in Scholarly Publishing

First posted on the ORIGINal Thoughts Blog

Kristen Overstreet
Senior Partner, Origin Editorial
Kristen@origineditorial.com
X: Overstrk
LinkedIn: www.linkedin.com/in/origineditorial
ORCID: 0000-0002-4417-3611

Take Home Points:

Peer review is the modus operandi of the scholarly publishing industry, with a purpose of increasing the quality and integrity of published content. There are multiple workflows for peer review, but double-anonymous peer review (DAPR) has been shown in numerous studies (examples here, here, here) to increase integrity by reducing bias. DAPR is the common peer-review workflow in the humanities and social sciences, beginning in the field of Sociology in 1955; it is used in many other fields as well. However, single-anonymous peer review (SAPR) is the most common workflow.  Other models of peer review include triple anonymous, open peer review, post publication, and others (a more complete list with definitions is included in a nice graphic here).

In recent years, a number of societies and publishers have converted their peer review to DAPR because they found it reduced the potential for conscious and unconscious bias in the peer-review process; thus, DAPR is gaining ground on SAPR. For example, the British Ecological Society switched to DAPR in 2023 following a study that showed improved objectivity in the peer-review process as well as making it easier to recruit reviewers when using DAPR.  In 2020, the Institute of Physics began the transition to DAPR based on demands from their community and a desire to ensure equity for all peer-review participants.  They received the ALPSP Impact Award in 2023 for being “the first society publisher to combine double-anonymous and transparent peer review.”  The Royal Society switched Proceedings B to DAPR as of January 2024, citing that recent studies show DAPR reduces reviewer bias; and the Journal of Bacteriology is currently piloting DAPR “to support fair and unbiased evaluation of submitted manuscripts.”  These few examples show that societies are willing to research the issue and make significant changes to their peer-review processes to improve integrity because they value equity.

Potential Benefits

Although it may require changing the peer-review model for your journal, the potential merits of DAPR are worth the effort and include:

• Reduced conscious or unconscious bias against authors based on factors such as gender, race, nationality, institutional affiliation, and prominence,
• Higher reviewer acceptance rate because reviewers are unaware of authors’ characteristics, such as gender, race, nationality, institutional affiliation, and prominence, and
• Higher quality reviews.

Research abounds that supports the use of DAPR to potentially reduce bias and increase equity. For example, Tomkins et al. found in their study “that single-blind reviewing confers a significant advantage to papers with famous authors and authors from high-prestige institutions.” Huber et al. performed a study where a manuscript was sent for peer review three times: with the prominent author identified, neither author identified, or the low-prominence author identified, and found that status bias existed when the authors were identified. Parmanne et al. found that there is a “power imbalance between authors and reviewers in single-blind review.”  They also found that overall review quality was significantly higher in a DAPR process, but the reviewer acceptance rate did not change. However, Charles Fox found in a study that looked at about 3,700 papers submitted to an ecology journal that reviewers were more likely to agree to review if they did not know the names of the authors, “reduc[ing] the average time from submission to decision by about 3.5 days.”

Challenges

Despite the advantages of DAPR, there are potential negatives, including:

• Difficulty identifying conflicts of interest, and
• Difficulty in achieving anonymity.

Researchers have also indicated concerns about the lack of transparency in a DAPR process and the potential difficulty of identifying conflicts of interest, and a number of studies discuss the difficulty of truly anonymizing authors’ identities.  For example, this study showed how an AI program reliably identified authors based on the creative identity and research domain revealed in the abstract and introduction, self-citations, and citation diversity. This article discusses “practical issues with double-blind reviewing,” including the ease of identifying author identity in smaller fields of study and the increasing practice of posting early versions on preprint servers. There is also an issue of maintaining anonymity with requirements to register clinical trials and systematic reviews.  Despite a journal’s efforts to ensure DAPR, reviewers can also intentionally search for author identities. This study found that “more than a third of [reviewers] self-report[ed] deliberately searching for their assigned papers online.”

DAPR can be more work for the authors and editorial offices who have to ensure submissions are properly anonymized. Journals using DAPR have different policies. The extent of the anonymization effort can be to the best of everyone’s ability involved in the process or simply left to the author to make the effort in order to realize the benefit. This means some journals do only a cursory check for anonymization, perhaps ensuring the title page is not included while others expend more effort to review and remove potential identifying information.

Worth the Effort?

Despite the potential difficulties of managing DAPR, if there is a chance that bias is reduced, isn’t it worth doing?  “. . . Occasional failures of anonymisation do not counteract the benefits of improved objectivity of peer review,” according to Fox, and the editor of the Journal of Bacteriology says “. . .there is an assumption that reviewers will simply try to guess who the authors are or scan through bioRxiv. To this I say, ‘Please don’t’ as you might guess wrong and base your review on bad information. More importantly, respect your colleagues’ wishes and the review process.” And this editor of science journals says, “Double blind peer review is . . . most effective when it is applied to all manuscripts, rather than offered as an option” to diminish the perception that authors in less privileged positions are more likely to choose DAPR. “Nevertheless, reviewers also have the choice to be unaware of author information, and thus to eliminate their own potential unconscious bias – an option they do not have when the author information is prominent on the first page of the paper.”

Published scholarly content should meet high levels of integrity, and that includes equity. This article published in 2012 stated that Africa, South America, and Oceania received less than 5% of the world’s citations, suggesting a negligible contribution to research, and this article indicated that geographic bias plays a role in knowledge diffusion.  The Institute of Physics has seen authors from Africa receive more than twice as many accept decisions since switching to DAPR. The effort required to anonymize a submission and educate reviewers about the importance of respecting DAPR is worth it to potentially decrease conscious and unconscious bias in the peer review process.

In Part 2 of this series, editorial office experts with direct experience operationalizing DAPR will share their insights on the effort required to anonymize a submission and educate reviewers.