Editing of reviewer comments

March 2020

Peer reviewers are asked to contribute intellectual work to assess and improve scholarly publications. As with all work, the quality and characteristics of peer reviews vary. Editors responsibilities include support not only to the peer reviewers who typically volunteer the time and knowledge but also to the authors, who reasonably should expect non-conflicted, thoughtful, unbiased, thorough reviews of the work in question and to not be subjected to hostile or personal attacks. This topic was discussed by participants of the COPE Forum, March 2020.

Editing of peer reviews

After the Forum discussion we shared an online questionnaire with members and non-members to gain more feedback on this topic. Survey results

Key points

Examples of possible problematic reviews or circumstances for which some editors might consider whether to edit or quash the review:

• "This author group clearly is lacking any fundamental knowledge of the topic."
• The reviewer recommends inclusion of their own work in the reference list without clear reasons.
• The editor encouraged the submission of the work and is eager to publish it, but one review is very negative.
• The review is replete with typographic errors. 
• The review is a single line "This paper should be revised" or "This paper should be rejected". 
• The reviewer accuses the authors of plagiarism or other misconduct within the body of the review. 
• The reviewer's comments are very different from those of the other reviewers and it seems that the reviewer did not understand the paper. 

Questions for the Forum

• Is it ever acceptable for an editor to change the content of a peer review or to quash it
  altogether?

• If so, under what circumstances would this be acceptable?

• If not, why not?

Comments from the Forum, March 2020

NOTE, Comments do not imply formal COPE advice, or consensus.

• We should differentiate between ethical screening and unethical censorship. If there are personal insults and ad hominem attacks on the author or investigator, it is appropriate to screen to maintain civil, courteous, and respectful discourse. However, if a peer reviewer provides rational, logical explanations and reasoning for the scientific argument, and the editor suppresses or censors it from publication to the readership, that is unethical. • It is not difficult to differentiate between personal attacks and principled discussion. Editors should not censor principled discussion. Editors should screen for personal insults
• It is appropriate to rescind or cancel comments by a reviewer if they are not constructive or if they are insulting, condescending and where there is be a conflict of interest. There is no value in sending comments that are insulting or unproductive to the authors.
• There should be a very clear difference between editing scientific comments and personal comments, specifically those that are outright sexist or racist.
• I would hope that any reviewer who expresses racist or sexist comments would not just be edited, but that their review would not be considered, and they would no longer be asked to review for that journal. They have demonstrated that they are incapable of objective judgement, and they should not be relied on for reviews in future. I would strongly urge COPE to issue guidelines along those lines.
• Journals should not be a conduit for personal attacks against authors; journals should be about the academic content.
• Where we need to be careful is when the editorial team or the editor-in-chief or associate editors do not agree in principle with the reviewer’s comments. If the reviewer’s professional opinion is well supported by their statements it should be accepted, regardless of the biases of the editors.
• I disagree with censoring reviewers. As an editor, I feel that you should be able to explain to a reviewer and to an author why you disagree with a reviewer's comment or perspective. The idea that you remove a reviewer's comment because you disagree with them on a scientific basis is wrong. Ad hominem attacks are a different category. Reports with ad hominem attacks should be returned to the reviewer for them to remove, together with a clear explanation from the editor why the report cannot be used if it contains hostile language. 
• Why would you edit out inappropriate comments? Why not return the report to the reviewer, explaining why the language they use is inappropriate and give them the opportunity to rewrite their review? Use this as an opportunity to educate the reviewer.
• There are different forms of censorship of the peer review process. Censorship pertains not only to peer reviewed manuscripts, requested by the editor of a journal, but also other forms of peer review, such as a letter to the editor or a comment for a correction.
• Many journals tell reviewers not to make a direct recommendation for the outcome of the paper they are reviewing. If they ignore that instruction and do make a direct recommendation, I will simply remove it and tell the reviewer, so there is a record.
• It is appropriate to reach out to reviewers and discuss strategies for future reviews.
• If changes to a review are made, or the editor chooses not to include a review, the editor should reach out to the reviewer first, to work with them and see if they can change their review to something more appropriate. If the reviewer is not willing to do so, the editor might explain to them why their review is not appropriate and acceptable practice. If a journal has a policy, it is then easy to reach out to reviewers and explain why their comments are problematic. Reviewers may not always be aware that some of the comments are problematic.
• Working with reviewers is important and editors can encourage them to revise their comments. Editors have a duty to educate reviewers. It is a learning process and reviewers often do not receive any training.
• Should we have a standard procedure where reviewers are notified if their review is edited or is not sent to the authors? Where should that process be documented to the reviewers? Should we have a statement in the instructions for reviewers, documenting the policy of the journal?
• We do a lot of reviewer training. We always reference the COPE guidelines. Also, reviewers are not decision makers; they are simply advisors to the editor.
• Many of the authors submitting manuscripts to my journal are new authors so I like to give as much positive, constructive feedback as possible. I have only rarely edited comments to put a positive spin on the review. I point out deficiencies and rationale for comments. Demeaning comments do not encourage good scholarship. I find overall that my reviewers are very respectful and phrase their review comments well. Each year at the editorial board meeting we discuss ways to do "good reviews".
• There is a disproportionate negative impact of hostile comments on women, non-binary people and members of underrepresented minorities. This is an opportunity to educate reviewers. The review should not be used unless the language is amended.
• We post our requirements for reviewers on our website stating that if you agree to review for us, you also agree to behave in a professional manner in all interactions with the author but also with the editorial team and the editors. We can then go back to reviewers who write reports that are inflammatory or have personal attacks and have a discussion with them in terms of how to write better reports in the future.
• Regarding helping to promote a culture of kindness in peer review, we have found it helpful to add a comment in the peer review score sheet itself, just ahead of the comment fields, to the effect "Please keep your comments clear, specific, and constructive, aimed at helping the authors improve their work".
• I am surprised by the number of unprofessional comments from reviewers that we see on a regular basis, especially in the current context where reviewer reports can be officially published and have a DOI.
• What happens if the editor edits the peer review report and it is then published and the author objects to the changes?
• Perhaps what the COPE community need to consider is having clear and explicit guidelines published with their publication ethics policies on what they do with reviewer reports that contain inappropriate language.

There are more comments on editing peer reviews posted on the COPE website before the Forum discussion.

Related resource

• Editing peer reviews, COPE guidelines
• Editing of reviewer comments survey, 2020
• Who owns peer reviews? discussion document and podcast, September 2017