Two of four reviewer reports received by the editor-in-chief of a journal contained a significant amount of verbatim textual overlap. Although of the same native (not English) language, the two reviewers are affiliated to institutions in different countries. The reports were submitted to the journal within 5 days of each other. Both reviewers suggested rejection of the submission.
Separately, contacted by the editor-in-chief, both reviewers categorically denied that they communicated with anyone (including any other reviewer) about the paper and their review work. One reviewer explained the similarities in the reviews as appearing “due to the fact that I have worked with [the other reviewer] as a co-editor for a special issue … a few months ago. To ease the review work, we prepared a checklist … . It seems that we are still making reference to this checklist when we review papers on the same topic, and this explains the similarities in the terminology and approach that we used in our … reviews. Moreover, I think that the convergence of our opinion about the overall quality, strengths and weaknesses of the paper is due to the fact that we are actively collaborating on the topic … and therefore have similar ideas about what it takes to provide a good contribution in this field of research”.
The other reviewer offered the following explanations: “[The other reviewer] and I have coauthored several papers … and, as a result, share many ideas about the ... literature; as a result of working together on several papers on the subject, we use the same perspective in assessing how a study fits into the literature and the contributions that it makes; potential overlaps in the terminology can be due to the fact that we developed for an old special issue that we guest edited a checklist for paper review …, that we shared with each other. Some of the common terminology in our review can be found in this checklist; … neither one of us is a native English speaker. This is why some expressions (common in … but unusual in English) appear in both of our reviews”.
Questions for the COPE Forum
(1) Should the editor-in-chief rely on the reports of the two reviewers?
(2) Should the editor-in-chief invite in the future the two researchers to review other submissions in the field?
The Forum suggested that the editor should consider that the explanations of the reviewers could be feasible and may in fact be true, particularly as their reviews were not dissimilar to the two other reviews of this paper in terms of content and recommendations. The editor noted that he has seen the checklist and that it does look useful and well designed, and could possibly be shared with other reviewers. In the end, it has to be a judgement call by the editor. Whether or not the editor decides to invite these researchers to review other submissions depends on whether he accepts their explanations.
One suggestion was to use plagiarism detection software on the reviewer reports and the special issue as a background check. This may give some indication of the overlap. Another suggestion was to allow the reviewers to continue to review other submissions but to monitor and score them so that their performance can be tracked over a period of time. The editor might also like to suggest that they refer to the COPE Ethical Guidelines for Peer Reviewers.
Polling the Forum audience in relation to who uses structured review forms and expects their reviewers to use them revealed that about a third do use them, but there was not much support among those not using structured forms to introduce this practice