Ethical considerations around watchlists

March 2024

Ethical considerations of using and maintaining watchlists

Watch the introduction to "Ethical considerations of using and maintaining watchlists" with Dan Kulp:

Watchlists are a contentious issue in scholarly publishing. They are used to maintain a record of authors or journals who have behaved unethically, either for internal publisher/journal purposes or as a public reference point. Several publicly available watchlists have been removed in recent years owing to concerns around the lack of transparency in the criteria they use. Others note the legal, reputational, and data protection risks involved in maintaining such lists. However, with the increasing scale of publication manipulation in recent years, some make a case that watchlists can be a useful tool in protecting the integrity of the published record, if handled ethically.

Questions for the Forum discussion

This Forum encouraged a discussion about the ethical considerations of using and maintaining watchlists. It offered a space to think about issues such as:

• The benefits and potential risks associated with watchlists
• What criteria should be used if watchlists are to be employed?
• What mitigations should be adopted, including the need for appeals processes and limitations policies, legal guidance, and transparency?
• The potential consequences of being added to a watchlist.

Comments from the COPE Forum, March 2024

NOTE, comments do not imply formal COPE advice or consensus

Introduction

• Watchlists have been used informally and principally privately by journals and publishers as quality control tools to identify individuals who have engaged in unethical behaviour. There have also been publicly available watchlists of individuals, journals, or publishers, such as Beall’s List (closed in 2017). However, these often create controversy over lack of transparency, reputational risk, and data protection. Cabell’s Predatory Reports are the latest iteration, but as a paid-for service they engage in deeper evaluation of journals and are transparent about the criteria they use.
• Now, the increasing scale of systematic manipulation is leading to more calls for information sharing to protect the published record.

Benefits and potential risks

• Watchlists of individuals carry more sensitivities than those of journals or publishers as they can have an impact on scholars’ reputation and professional future, as well as issues over sharing of personal information. Watchlists of journals can be made public with reduced risks, although there have been cases where journals have tried to sue over their inclusion on such lists to mitigate reputational damage. More attention may need to be paid to intent and severity for individuals; for journals processes and modes of operation are more relevant. There is more scope for addressing individual misconduct than there is for journals.
• There are also now many ‘safelists’ and benchmarks used by agencies to emphasise trust. These include Think.Check.Submit, and membership or inclusion by DOAJ, Web of Science, OASPA, ICMJE as well as COPE.
• The greatest risk to the use of watchlists concerns data sharing and privacy, for example, under the General Data Protection Regulation (GDPR).
• Those using watchlists, even privately, should take their own legal counsel on this and related issues, especially if they plan to use them to invoke sanctions or impose barriers to publishing.
• Watchlists can only alert users to look for suspicious behaviour. Subsequent actions should be considered carefully, for example, more scrutiny in peer review.
• Acting together to identify unethical individuals, organisations or journals gives weight to the endeavour to protect the scholarly record and can be a positive step towards disrupting paper mills and peer review rings. A cross-industry response would also reduce the incidence of cases where journals keep suspicious articles in the submission system in order to prevent them being submitted elsewhere. However, caution is needed to avoid revealing too much about the criteria used, which could help paper mills to change tactics.
• A key challenge is to share information ethically in a way which allows journals and institutions to identify suspicious behaviour in advance of submission.
• The STM Integrity Hub’s Duplicate Submission Tracker is a tool to identify when authors submit the same paper to more than one journal or publisher. The Integrity Hub also has a working group exploring the use of watchlists for serious integrity issues and hopes to launch a pilot later in 2024.
• There may be useful models from healthcare or finance where similar risk factors have been addressed.
• Journals and institutions should make their points of contact for integrity issues clear in order to promote constructive conversation and transparency.
• Should all authors be included on a watchlist when integrity issues are identified, or solely the corresponding author? COPE advises including all coauthors in all communications about an article to ensure that they are aware and have an opportunity to respond.

Mitigations

• Individuals should be notified if they have been added to a watchlist and should have the right to appeal the decision.
• Attention should be paid to how watchlists operate alongside systems of double or triple anonymous peer review, or pre-preprint review systems such as Publish, Review, Curate.
• Editorial decisions must be transparent and backed up without reference to a secret list.
• Publishers could develop a credit score system for authors as an alternative to watchlists, building on initiatives like ORCID trust markers and the STM Researcher Identity Working Group. However, more work would be needed on the impact on credit scores of different types of retractions (for example, author-initiated retractions versus those for misconduct), and on the status of new researchers.
• Watchlists should be made available only to those in specified roles, such as integrity staff.

COPE does not recommend the use of lists which prevent anyone from accessing opportunities to publish, and advises that all parties consult legal counsel for individual cases.

COPE is exploring the need for guidance on ethical issues surrounding watchlists and welcomes comments on this topic.