A recent survey in which we asked COPE members to vote on the diversity, equity and inclusion topics they would like to discuss, bias in peer review was voted the topic of most interest, so we are devoting this COPE Forum discussion to it. While this important topic aligns most closely with COPE Core Practice: Peer Review Processes, its impact is wider reaching. The explicit and implicit biases, or conscious and unconscious biases, of editors and peer reviewers could prejudice which manuscripts are reviewed or ultimately accepted for publication. As COPE member Cambridge University Press explains in their information for peer reviewers: “Reviewers must give unbiased consideration to each manuscript submitted. They should judge each on its merits, without regard to race, religion, nationality, sex, seniority, or institutional affiliation of the author(s). … Rooting your review in evidence from the paper or proposal is crucial in avoiding bias.”
Diversity and inclusion in peer review survey
Diversity and inclusion initiatives seem to be key to helping reduce bias in peer review. As part of Peer Review Week 2018, COPE conducted a survey to understand more about how the community views this topic. Recently, we wondered if and how attitudes, policies, and/or practices have evolved over the past few years. To that end, COPE largely reproduced the survey during Peer Review Week 2021, and below are highlights of the responses.
COPE received 267 responses to the new version of the anonymous survey. The overall profile of respondents was similar in both versions, with 60% of the respondents being journal editors and 18% working for publishers in 2021. Perhaps it is, therefore, not surprising that the responses to several of the questions asked were nearly identical to the previous survey; these included how important diversity and inclusion in peer review, to whom diversity and inclusion apply, and definitions of peer review diversity.
COPE received notable responses, however, to the question about the extent to which one’s publication/organisation has achieved diversity. From 2018 to 2021, respondents reported a 69% increase in journals/publishers providing in-house training to promote diversity and inclusion in peer review, as well as a 19% decrease in respondents who thought their journal/publisher had achieved the ideal level of diversity in its peer reviewer pool. Two new reply options to this question also garnered significant responses, with 69% of journals/publishers actively working toward increasing the diversity in its peer reviewer pool and 29% of respondents reporting their journal/publisher had achieved an ideal level of diversity in its peer reviewer pool.
Detailed below are other significant shifts in response to the kinds of diversity seen as being important in peer review. Two of the fifteen new reply options to this question in the 2021 survey also garnered significant responses, with 67% of respondents indicating that Career Level was important and 63% indicating that Geographic Diversity was important.
- The importance of Expertise Areas/Specialities decreased by 27% (from 88% in 2018 to 64% in 2021).
- The importance of Sex/Gender Identity increased by 14% (from 77% in 2018 to 88% in 2021).
It is also worth highlighting some of the free-text responses received. The replies to a question about what changed with respect to diversity, equity, and inclusion in their editorial policies/practices or organisation over the past 3 years ranged from “nothing” to “unconscious bias training.” Finally, several respondents provided general comments, including:
- “Diversity must be promoted and achieved without compromising the integrity of the process.”
- “Diversity an [sic] inclusion in the process is essential in a global and diverse world.”
- “The key issue in peer review is fair objective evaluation of submitted work.”
- “Only knowledge and qualifications matter”
- “We may never have a fully representative review panel but at least we can understand their biases.”
These survey results indicate an overall positive change over the past 3 years in attitudes, policies, and practices about diversity and inclusion in peer review. Not surprisingly, however, there is still more for publications and organisations to do. Meaningful and lasting change takes time, of course, and we want to thank those who not only participated in this survey but also contribute to ongoing efforts to make the peer review process more diverse, equitable, and inclusive for everyone.
Questions for discussion at the Forum
With the ultimate goal of eliminating bias in peer review, we hope you will share during the COPE Forum some of the practical steps you are taking to reduce bias.
- How does your publication/organisation attempt to manage the effects of bias in peer review?
- How is your publication/organisation working to invite a more diverse range of editors and peer reviewers?
- Does your publication/organisation ask peer reviewers to ensure not only that the correct references are cited but also that the references reflect a diverse range of authors?
Following the discussion on bias in peer review, the Forum will hear from members who have submitted cases for discussion and advice. The Forum agenda includes links to the new cases.
Anyone (members and non-members) can comment on this discussion.
Comments are reviewed and, on approval, added below.