Dealing with complaints about the integrity of published research

June 2022

Watch the introduction to the topic "Dealing with complaints about the integrity of published research" with COPE Council member, Jigisha Patel.

COPE Forum discussion

Between 2013 and 2015, COPE produced a number of documents on how to respond to the increasing number of people who were raising complaints about research integrity in journals, and more recently a COPE Forum dealt with how to coordinate complaints amongst multiple journals. Since then, complaints about integrity in research publications have become even more prevalent and the issues that they identify have become more complex. It is, therefore, timely to revisit these documents, and update the guidance.

COPE has produced a draft discussion document on "Dealing with complaints about the integrity of published research". We would like comments on this document from all interested parties, and encourage journal editors and publishers to comment (whether or not they are COPE members), and also welcome comments from researchers/authors and academic institutions.

Questions for discussion

Are there aspects of the guidance in the discussion document that you feel need clarification or change?
Are there aspects of complaints handling which you do not feel are adequately covered in the document?
Are there any other tools or resources that you would like to see around this subject?
What policies do journals have on responding to complaints that are not made to them directly (for example, on social media, PubPeer)?
Do complaints raised via different channels present different challenges?

Forum discussion summary is not considered formal COPE policy but may be used to develop other types of COPE guidance.

Comments from the COPE Forum, June 2022

NOTE, Comments do not imply formal COPE advice, or consensus

Journals need to be explicit about timelines and set expectations for complainants. It is important to keep complainants up to date. If expectations are set upfront, then complainants can be told if they are not respecting those boundaries.
If an editor is receiving multiple emails, they need to be clear about time frames. It is important that editors always respond to complainants, but they should try not to get too caught up in the issue, and it is not necessary to respond to all communications. If editors encounter harassment, they might decide which staff member can manage those emails and have a process for dealing with them. A strategy for how to manage these emails so that it does not overwhelm the editorial office is useful.
Journals can make it clear to complainants that the journal is conducting an investigation, but setting a timeline as to how long it will take to resolve the case can be tricky. It can be hard to estimate how long a case will take to resolve, especially if the outcome is dependent on institutional input. Some editors will not respond continually if they have no information to share, if the case is ongoing.
For cases with persistent complainants, it can be helpful for journals to define a schedule for when updates will be provided, whether that is once a month or once a quarter, or only at the end of the case. It is important to set expectations so that complainants know that the journal will not be replying to weekly messages.
It is also a good idea to stay customer focused and tell the applicant/informer the investigation is ongoing periodically.
Some journals provide a 1 month timeline for response in their initial acknowledgment.
Some complainants post on Twitter and accuse journals of missing deadlines. Guidance from COPE that editors could refer complainants to would be useful. Editors should be cautious about providing guarantees and time lines because investigations can be unpredictable. PLOS has some guidelines on handling difficult cases. The section on "Why does it take so long to resolve publication ethics cases?" is helpful in explaining to complainants why a three month period to action is actually very fast.
There are a lot of great resources for journals/publishers but having a resource for editors to direct the "people who raise complaints" to could be helpful. COPE has flowcharts on Responding to whistleblowers when concerns are raised via social media and Responding to whistleblowers when concerns are raised directly.
The volume of complaints via social media can be huge. The editorial team need to decide is the complaint valid? Does it have merit? The journal should always acknowledge the complaint if posted online. If the complaint has merit, ask for an email address and more information and then take the issue offline. If possible, have a person or team responsible for dealing with complaints via social media.
Some journal use a central email account and avoid exposure of specific staff members where complainants have targeted particular people.
A single point of contact in cases of persistent complainers from a central inbox is critical to filter communication appropriately. Timelines can be difficult to set because in many cases editors are relying on replies from other external parties or need to engage in follow-up.
It can take a lot of time to determine which complaints are credible, especially comments received via social media. Some journals categorise complaints based on when the article was published. If the article is <6 years old it might be prioritised over articles older than >6 years. The US National Institutes of Health (NIH) has a data retention policy of 6 years.
Journals should consider the seriousness of the complaint rather than how long ago the article was published. A paper will image manipulation published 10-15 years ago, for example, would be considered high priority. Editors should also prioritise on the type of paper (research articles having a higher priority).
How should harassment be defined? Complainants often bombard journals with emails while they are investigating an allegation, and say that the journal is too slow in resolving the case.
Regarding PubPeer, many journals do not actively look at comments, unless they are related to an ongoing investigation and they are brought to the attention of the journal via social media.
Do journals have a formal policy or process for handling complaints made on social media? Some journals have general communication policies that can apply to resolving matters in a public forum by means of a crisis communication plan, but nothing more specific.
Some publishers acknowledge that they do not specifically search for concerns raised on social media, but when publication ethics concerns come to their attention, they acknowledge the concern and open a publication ethics case for the publication ethics team to look into (or link the tweet to an existing case if applicable). In some cases, the publication ethics team might contact the individual who raised the concerns on social media to discuss the matter in more detail first.
How do you decide when it would be appropriate to add a link to PubPeer or Twitter in a journal notice? If there is a relevant PubPeer comment, should you always include a link?
Regarding social medial complaints, PLOS’s policy notes that readers should email them directly with concerns/complaints: "Do not rely solely on posts to blogs, social media, or other third-party websites to make us aware of concerns." However, PLOS do follow-up if they become aware of integrity issues raised on social media but do not typically link to social media in published editorial notices.
COPE decided not to use the term whistle blower in this document because it was felt that the phase was colloquial in nature and not well understood by those whose first language is not English. COPE wanted to use a neutral term and whistle blower can have negative connotations.

ACTION: COPE will consider the comments from the Forum and add advice to the discussion document about timelines, seriousness of the complaint, type of article, and managing high volumes of comments.

We welcome your comments on the discussion.

Related resources

Authorship and politics COPE position statement

A frank discussion on handling difficult cases PLOS blog February 2022

A draft COPE discussion document

Introduction

In 2013, the Chair of COPE, Virginia Barbour, wrote a document to stimulate discussion on how to respond to anonymous whistleblowers. This was in response to the fact that an increasing number of people were raising complaints anonymously about research integrity in journals.

In 2015, two flowcharts were published jointly with BioMed Central: on responding to whistleblowers who raise concerns directly through email or other forms of correspondence to the journal, or to those who raise concerns through social media. Some people raise a series of complaints to the same journal, and guidance was provided for that scenario. More recently, a COPE Forum was held on the issue of coordinating such complaints amongst multiple journals.

Since these documents were produced, reporting of ethical complaints about research publications has become even more prevalent and the issues that they identify have become more complex. This includes situations where multiple complaints are raised simultaneously to multiple journals.

It is, therefore, timely to revisit these documents, and update the guidance. This document aims to do that. It is currently a draft which we intend to share with interested parties to solicit views and comments. We encourage journal editors and publishers to comment (whether or not they are COPE members), and also welcome comments from researchers/authors and academic institutions.

In response to changes in the terminology being used in scholarly publishing we refer to “people who raise complaints” rather than whistleblowers. Previous documents referred to whistleblowers, and these have not been changed.

Who raises complaints?

This guidance refers to anyone, named or anonymous, who notifies journals or publishers of (a) unsound, unethical, or otherwise concerning published research; (b) research reporting issues; (c) publication ethics complaints; or (d) other matters that have implications for the integrity or reliability of the published research record. When complaints are raised, evidence should be provided to support the claims. Some researchers spend much time detecting and reporting on supposedly fraudulent activities and may raise complaints over many publications.

Types of issues reported

Typical issues reported include (but are not limited to) allegations of plagiarism, image manipulation, data appropriation (use without permission), data fabrication, various types of errors, authorship issues, undisclosed conflicts of interest, computer generated manuscripts, lack of ethics approval for human subjects research, concerns about study design or results reported for animal research.

How are complaints raised to editors and publishers?

There are multiple routes to alert editors and publishers to what has been uncovered. It is now possible for issues to be aired to the whole world at once rather than privately, not only via classical media, but also through social media outlets such as Twitter and through websites such as PubPeer which allow users to discuss papers post-publication. This guidance recommends that journals should be contacted directly with any complaints. This way the person raising complaints can be sure that editors have received the information, and editors will have a direct channel of communication with the person raising the issue. This will enable editors to seek any needed clarifications and notify the correspondent of the outcome of the case. In addition to notifying the journal directly, concerned parties are welcome to use other forums (e.g. Twitter, Pubpeer) to discuss complaints and notify readers of potential issues regarding published work, but such forums should not be relied upon to reach affected journals or publishers. Journals should have policies in place regarding how/whether they will respond to issues that are not raised to them directly.

How should complaints be responded to?

Please refer to the flowcharts on responding to whistleblowers who raise concerns directly through email, or to those who raise concerns through social media, as well as the guidance on sharing information on possible misconduct. The roles of each party involved in handling these complaints (Editor/Journal, Publisher, Institution) are set out in the CLUE Guidelines.

All requests should be considered, and investigated if they have credibility and are about research integrity in a publication.
Acknowledge receipt and investigate according to the appropriate COPE flowchart or guidance, and the publisher’s guidance.
Responses should be neutral and fact-based, indicating the intended action. For example, “Dear XX, Thank you for getting in contact with us and drawing our attention to the concerns you have. We will investigate and take action as needed, in accordance with COPE guidelines. Yours sincerely”. Do not get drawn into personal exchanges.
There is no need to engage in extensive correspondence. If this process has been followed and there is still concern, the advice of COPE can be sought.
Complaints of a harassing, offensive, threatening, or defamatory manner should be referred to legal counsel or other appropriate authorities and the journal should notify the complainant that allegations made in such language will not be investigated.
After investigation, if it is found that the complaint has substance, follow the appropriate COPE guidance for dealing with the issue.
It is important to respect the correspondent’s right to anonymity, and their identity should not be divulged to the party facing allegations without the correspondent’s express permission
Consider thanking the person raising the concern (anonymously if appropriate) in any journal notice that may be applied, e.g. retraction. The notice could link to any comment in the public domain (PubPeer, Twitter, etc) if appropriate.
Once the investigation is closed, inform the complainant of the action you are taking. If they do not accept your response and you are confident in it, it is fine to reiterate this once and then say you consider the matter closed and not respond further.
COPE does not respond when copied into complaints.

Dealing with multiple complaints to the same journal from one individual

On occasion a journal may get several complaints from the same source. Complaints may be directed at an author, an editor, or the journal in general. If these complaints turn out to be well founded, investigations should proceed as warranted. However, there are also cases where an individual makes repeated allegations against a journal, editor, or author that turn out to be baseless. COPE has provided separate guidance on how to respond in these circumstances.

Dealing with issues raised across multiple journals

Complaints raised can develop into large investigations with multiple journals, editors, and publishers involved. These investigations are becoming more frequent and therefore COPE is considering updated guidance on how editors and publishers can work together to resolve these complaints without breaching ethical considerations. An initial discussion was held at the COPE Forum in June 2021: Coordinating multi-journal complaints.

Your comments

We welcome comments on this discussion from members and non-members.

You must be logged in to leave a comment on the COPE website. If you are not a member of COPE you can register to create a non-member account.

Comments will not appear immediately. We review comments on our website before publishing them, to ensure they are respectful and relevant.

Comments

Posted by Shaun Khoo, 22/6/2022 8.22am

Thank you for the opportunity to respond to the draft discussion document. This response is being provided on behalf of the UNSW Sydney Conduct and Integrity Office who are responsible for research integrity matters. Within UNSW, the role of the Conduct and Integrity Office is to investigate potential breaches of the UNSW Research Code of Conduct, which implements the Australian Code for the Responsible Conduct of Research. Our response to the draft discussion document is below and we can be contacted at [email protected].

1. Are there aspects of the guidance in the discussion document that you feel need clarification or change?

The role of institutions is unclear. Under “Who raises complaints”, a complainant is defined only as someone who “notifies journals or publishers”, but broadly speaking, can also include someone who notifies institutions. In many cases, a complainant will notify both journal and institution, whilst in others the complaint will only be submitted to the journal or to the institution.

It is also important to note that complaints can be raised to editors and publishers via institutional research integrity offices. It would be helpful for the document to also address how the journal editors and publishers could assist or cooperate with the investigations conducted by institutions. This may happen when one of our research integrity officers contacts a journal or publisher making an enquiry about a paper or an investigation conducted by the journal, as well as when they notify a journal or publisher of the investigation outcome and request corrections or retraction of a paper. Quite often we either don’t receive a response, or a politely refused cooperation/advice where the two bodies receive the same complaint.

The relationship between journal and publisher processes and institutional processes could be strengthened. For example, the guidelines on “Sharing of information among editors-in-chief regarding possible misconduct” should automatically loop in institutional research integrity offices. The research integrity team at UNSW is very rarely approached by journals or publishers during their investigations, despite having contact details published on our website. Moreover, COPE guidelines don’t recommend that an institution be contacted unless the authors have not responded or provided an unsatisfactory response (e.g. COPE Flowchart – Fabricated data in a published article, https://doi.org/10.24318/cope.2019.2.4). Guidance regarding contacting institutional research integrity offices should be consistent across complaint types. We would recommend that publishers contact the Institutions Research Integrity Office early, as it is quite likely that the institutional research integrity office is also aware of the concerns and there may be concurrent processes unnecessarily underway.

2. Are there aspects of complaints handling which you do not feel are adequately covered in the document?

The guidance document needs to provide clear instructions on how to contact journals or publishers with research integrity concerns. We often find that journals do not have published contact details for research integrity issues. While some publishers have central research integrity mailboxes, this is not the case for all COPE members. Clear guidance that research integrity contacts should be readily available on journal websites would significantly aid us in making enquiries and requesting corrective action. This is particularly important where the content of correspondence is confidential in nature or where we are required to discharge certain obligations in relation to corrective action.

3. Are there any other tools or resources that you would like to see around this subject?

It would be great to have some guidance for institutions in reporting and escalating matters with journals and publishers. For example, it is unclear what path is best for us to take in escalating a matter when corrective action requested from a journal has not occurred in a timely manner. Guidance from COPE on how institutions can escalate matters would be beneficial and facilitate accurate information being provided for institutional investigations and appropriate corrective action following determinations. Of additional use would be guidance on appropriate pathways where a journal rejects an adverse institutional finding and associated request for retraction/correction, either on its own or because the co-authors will not agree to the corrective action when permission is sought by the journal.

4. What policies do journals have on responding to complaints that are not made to them directly (for example, on social media, PubPeer)?

UNSW as an institution manages any concern related to the integrity of our research that is brought to our attention. As long as a complaint contains sufficient information regarding a potential breach of our Research Code by a researcher at our institution, we will ordinarily consider the complaint. Once the complaint is accepted, it is handled the same as any other matter under our own procedure.

5. Do complaints raised via different channels present different challenges?

Complaints raised via different channels do sometimes present different challenges. For example, if a complainant is anonymous and cannot be contacted, it may not be possible to contact them for clarification or further information so as to proceed with the complaint. Complaints raised via PubPeer also tend to involve large numbers of papers, multiple respondents, sometimes opposing views, and take significant resources to investigate and resolve under a formal investigations structure. However, most of these challenges are not intrinsic to the medium by which a complaint is raised. It is certainly possible for a named complainant to cease responding to correspondence during the complaint handling process or for a complainant to privately and confidentiality send us allegations relating to many papers and respondents.

You are here