You are here

Home Resources Seminars and webinars

Webinar 2022: Managing paper mills

Practical steps for managing paper mills

September 2022

During Peer Review Week COPE hosted a webinar for journal editors and publishers on the practical steps to take in managing paper mills. Speakers described their experience of managing paper mills affecting their journals, with practical tips on how to manage systematic manipulation of the publication process, at scale.

Watch webinar

Back to top

Introduction: Manipulation of the publication process (paper mills) - guidance on dealing with large cases

The session was moderated by Jigisha Patel, opening the session with an update on some of COPE's activities around paper mills. Jigisha summarised the recommendations from the COPE and STM research on paper mills undertaken in 2022. Jigisha gave an update on the recently revised Systematic manipulation of the publication process which provides detailed guidance on what paper mills are, what types of evidence to look for and how likely the evidence found represents paper mill activity. Four major areas remain where editors and publishers need further guidance from COPE:

Areas for further guidance

1. How to deal with the practical and administrative burden of collecting the evidence.

2. How to provide timely due process that balances the series-level and article-level assessment, while also considering the resource impacts and constraints for journals/publishers.

3. The scope of and mechanisms for cross-publisher information sharing.

4. How to manage the conflicting demands of editorial freedom, publisher responsibility and legal risks.

New guidance aims

The new guidance aims to encompass the following general principles:

  • The need for journal policies that specify what actions will be taken if there are suspicions of manipulation of the publication process (paper mills).
  • The need for confidential information-sharing across publishers (e.g. via the COPE Publisher’s forum and/or STM Integrity Hub).
  • The management of cases en masse rather than on a case-by-case basis.
  • Transparency in retraction notices related to manipulation of the publication process.
  • Editorial decision making based on all evidence (including circumstantial) relevant to the series-level concerns, and empowering editors to apply judgement as to whether primary data or other documentation should be requested.
  • Publisher roles in supporting editors to issue decisions based on trustworthiness of content, and ensuring there are systems in place to support editors in responding to legal threats.

A publisher's perspective of identifying and investigating paper mill concerns

Time on recording : 8:25-18:10

Renee Hoch, Managing Editor leading the PLOS Publication Ethics team, spoke about managing of paper mills from the perspective of PLOS with a review of the tactics used by paper mills around content, authorship and peer review and how PLOS identify and investigate manipulation of the publication process.

A recent case, investigated by PLOS, was sparked by a red flag to do with author behaviour. Renee explained the steps taken in the investigation and what they uncovered:

  • The evaluation of a series of submissions linked to the author revealed a co-publication network involving authors and editors.
  • Editorial assessment revealed common issues across the series pertaining to authorship, competing interests, and peer review integrity.

PLOS concluded that retraction was warranted for most articles with links to the prolific author-editor network. In these cases the authors may attempt to republish the retracted articles elsewhere so it is essential to have good communication across publishers to effectively combat these issues.

Back to top

Paper mills and contract cheating: systematic enablers and recommendations

Time on recording: 19:05-33.20

Sarah Elaine Eaton, COPE Council Member, and Editor of the International Journal of Education Integrity, explained the parallels between student contract cheating and paper mills, and how some of what we know about contact cheating can be transferred to our understanding of paper mills.There has been around 15 years of scholarship around contract cheating, to understand how it works and its impact on higher education. Sarah says that if graduate students engage in contract cheating, that it could be a precursor to scholarly misconduct after graduation, and some of the historical and current understanding of contract cheating might help to understand more about paper mills.

In summary:

  • We need more collaboration between academic integrity researchers and scholars of publication.
  • That there could be links between contract cheating and paper mills considering the rates of repeat offences in contract cheating.
  • That there is a role for quality assurance bodies in the quality of publications produced at research universities.
  • Where legislative bodies act in contract cheating, consider extending this to paper mills.
  • That there is a role for COPE in education and advocacy around tackling paper mills.

Back to top

Practical tips on managing paper mills

Time on recording: 33:36-43:10

Jigisha Patel, COPE Council Member and independent research integrity specialist began with a reminder of who runs paper mills, how they work, and how there has been increasing interest in and development of technology in the past year to help detect paper mills. Jigisha recommended how to determine what information to gather to help identify manipulation of the publication process both from the authors and their institutions. She went on to suggest how to avoid publishing manuscripts from paper mills by looking at the vulnerabilities in the journal publishing process.

  • Be aware of how paper mills work.
  • Know what to look for in individual manuscripts, publication workflows and databases.
  • Keep an eye on what happens throughout manuscript tracking including after editorial acceptance.
  • Screen across journals and submissions.
  • Monitor what you publish eg audits.
  • Make sure your peer review process is secure.
  • Make sure guest editors are aware of and comply with editorial standards.

Back to top

STM Integrity Hub update

Time on recording: 43:30-53:55

Joris van Rossum, Director of Research Integrity at STM gave an update on the STM Integrity Hub in relation to technical solutions, which are increasingly important to protect research integrity, to help manage paper mills.

  • Collaboration is key in establishing standards and solutions, and corpuses of training materials. smaller publishers and journals must have equal access to the tools and resources.
  • 3 working groups at STM to develop tools that are a work in progress:
    • building a paper mill checker tool so editors can upload submitted manuscripts and check tem for various signals (soft launch Oct 2022)
    • duplicate submission tool (launch 2023)
    • image alteration and duplication

Back to top

Related resources

Audience questions

Following the presentations the audience asked the speakers questions, some of which were answered during the webinar. 
Time on recording: 54:14-1:21:00

The speakers very kindly answered questions we didn't get to on the day:

Can you give examples of competing interests that gave rise to suspicion [of paper mills activity]?

When evaluating a series for potential paper mill concerns, we are typically looking for concerns that link articles across the series. For competing interests, relationships between authors, editors, and/or reviewers present red flags. For example, have the authors and editors/reviewers published with one another, do they have the same affiliation currently or did they previously work at the same institution?

Can you give some more insight into the contribution of the peer review process to paper mills? How do they work? Any particular examples?

The peer review process can be used to help paper mill articles through to publication by:
(a) involving falsified or fake reviewer accounts, e.g. reviewers for whom the email addresses provided to the publisher are managed by the paper mill rather than the individual whose name and affiliation are listed;
(b) having individuals linked to the paper mill join editorial boards from which they can guarantee a positive peer review outcome, and/or
(c) having authors request these paper mill-linked editors and/or fake reviewers at the time of submission.
The handling editor, paper mill, or even the authors themselves (using a fake reviewer account) may provide the reviews so that ultimately the sources of review comments are misrepresented to the journal, the paper mill’s vested interest in peer review outcome presents a major competing interest impacting the review process, and the article does not receive a rigorous and objective evaluation. An example of this was discussed on Retraction Watch this week.

What is the most effective identifier of a paper mill [from the list cited in Renee Hoch’s talk]?

I encourage editors evaluating articles for paper mill concerns to take a thorough approach and evaluate submissions across multiple domains when first investigating a potential paper mill cohort. Paper mill series articles typically have one or more element(s) in common, with the commonalities extending beyond similarities an editor would expect based on the nature of the study, article type, and submission standards. However, the nature of the common element(s) varies from series to series: an identifier effective for identifying articles in one series may not apply to others. 

Bear in mind that when publicly discussing cases involving potential manipulation of the publication process, we need to strike a balance between raising awareness of the issue and not giving perpetrators of misconduct information that may help them evade our quality control processes. With this in mind, confidential discussions between journals or publishers are often best suited for discussing specific details about a paper mill series or concern. 

What is the difference between contract cheating and research collaboration?

Contract cheating is the outsourcing of academic work, often without the knowledge or permission of the instructor. Presumably, in a research collaboration, the principal investigator would be aware of — and approve — the collaboration.

What resources are there for multicultural researchers (i.e. help with writing) that would avoid paper mills?

Some publishers offer language editing services, and others may recommend copyediting or language editing service providers upon request. When seeking a company to help with language editing, avoid companies that offer help with peer review or article submission, or that request or offer authorship. Note that language editing does not suffice to meet ICMJE authorship criteria for research publications.

Where is the overlap between paper mills and predatory publishers?  Or are these considered separate but equally concerning issues in the peer review and publishing processes?

They are probably equally concerning. They are both bad in terms of contaminating the published record, but they operate in a slightly different way. And paper mills and predatory publishers can overlap eg predatory publishers may run paper mills. We need more research to understand this.

Are there any legitimate reasons for adding an author after acceptance?

We have seen cases wherein a colleague was excluded initially because of an honest error, a disagreement or change in authorship agreement among collaborators, or because an individual could not be reached during the submission and peer review process (e.g. due to field work, illness, or an unexpected prolonged absence). On the other hand, we have also received post-accept requests to add authors because of their involvement in APC payment; payment of article fees does not satisfy ICMJE authorship criteria.

I note that there are publishers who are listed as partners to the STM Integrity Hub who have also published retractions. How does the STM Integrity Hub consider this issue?

The number of retractions per publisher is expected to follow the number of total publications per publishers, so in that respect, it is not surprising the larger publishers have a high number of retractions. The Integrity Hub is working with publishers towards filtering out problematic papers at the point of submission.

What is involved in joining the STM Integrity hub initiative? What is the role/responsibility of partners?

Publishers as well as submission systems and publishing platforms can join the initiative by participating in one of the working groups or task forces. Participants are responsible for setting the strategy and priorities of the Hub, sharing information, testing our solutions, and creating policies and frameworks.

Might the competition between journals/publishers hinder the work of the Integrity Hub when it comes to sharing submissions?

Publishers feel protecting research integrity affects the entire scholarly ecosystem and is therefore a shared responsibility. Moreover, submissions in the Hub are never shared among publishers, but the Hub allows submissions from multiple publishers to be screened simultaneously. This happens without confidential information being shared with anyone.

Is there a good tool for identifying submitted publications that are being retracted or rejected?

Yes, for example Problematic Paper Screener Integrating this tool in the hub is one of our priorities.

Would you include suspected gift authorship in unusual behaviour?

Yes, gift authorship is not good practice.

Does offering manuscript preparation services for a fee to potential authors (as offered by many large companies globally), fall under the gamut of “Contract Cheating”? Would a declaration by authors that they sought assistance and a declaration by the provider, circumvent “Contract Cheating”

It is best practice to declare third party support received in conducting or reporting research, including in manuscript preparation and/or submission. However, declaring this support will not protect authors from actions on their articles if it comes to light that a company whose services were obtained was involved in manipulation of the publication process. Offering manuscript preparation services for a fee is not in isolation cause for concern, and is not in itself evidence of contract cheating or paper mill activity. Still, researchers should do due diligence in evaluating the reliability and integrity of these services, request details of the services provided, and avoid using services that guarantee positive peer review outcomes, recommend/request specific editors or reviewers, or are involved with the manuscript after its initial submission. The authors are ultimately responsible for the contents and for addressing any questions from the editors and reviewers.

What is the meaning of the term ‘guest editor’?

This is when an academic who is not a member of staff for the journal or publisher or a usual editor of a journal is invited to oversee the selection and peer review of a special series of submissions for a supplement or special issue for the journal. The guest editor is given responsibility for peer review and, if they are involved in a paper mill, they will exploit their role as guest editor and allow the publication of manuscripts in exchange for a fee.

Are there some disciplines more vulnerable to receiving paper mill submissions?

Medicine and biomedicine were the first fields to be recognised as being targeted by paper mills, but they can affect any field. 

Can any of the speakers name at least one name of someone identified to be positively involved in Paper Mills?

Often we don’t know who is behind a paper mill because we can only detect the authors who are taking part. Sometimes it is difficult to know whether the authors knowingly or unwittingly took part in a paper mill. Also, there can be legal barriers to naming individuals in any type of misconduct case.

How can I check whether an editor at one of our journals has been asking for publication fees from authors (publishing with us is free)?

It is difficult to comment on a specific case without knowing all of the details. In this case it seems you have two major concerns; the conduct of the editor and the quality of the submissions in your system. In many cases, it is not possible to prove that an editor took a bribe or committed misconduct in any way.  You may just need to remind them of your expectations and what they can and can’t do and then monitor their activity. For your submissions, you may want to check whether they underwent peer review, the appropriateness of the peer reviewers and whether peer review reports and editorial decision making were appropriate. This would give you some indication of whether the editor was allowing submissions through inappropriately. 

 Is there a database of known or suspected paper mill authors?

Not to our knowledge and it is likely that such a database would contravene privacy laws as well as potentially cause reputational damage to individuals who work ethically, but who happen to have the same name as an individual in the database.

What are the ways in which paper mills ensure publication of their products?

They find ways to circumvent the usual quality control processes of a journal to ensure that publications will pass peer review and be published. Details of how this is done are provided in the new COPE guidance.

Is there any consequence at the moment for authors involved  paper mill chains?

Journals can report or express concern about author activity around paper mills to the authors’ institutions. The institution decides whether there are any consequences.

Our legal team feel strongly that we should put all allegations & evidence of paper mill activity to the authors in case there is a reasonable explanation. They insist this is part of a fair process of investigation. I have seen many paper mill investigators warn against this practice, as we can alert paper mills how to avoid detection. What are the views of the speaker on this?

It is fair practice to put allegations and evidence of paper mill activity to authors. They should be given an opportunity to explain. We don’t need to share how the evidence was found - eg what screening methods or tools were used.

When we use a plagiarism software is there a rate of duplication which indicates plagiarism?

There should always be a human assessment of plagiarism. There is no cut off percentage that can be considered acceptable or unacceptable.

Editors in developing countries do not always have access to lots of technology. What are the suggestions for them?

There are standard checks that are available within certain editorial systems. Collaboration with the STM Research Integrity Hub may also be helpful.

To what extent is it acceptable to add authors to a manuscript, if at all?

At revision, you would only expect additions if they are requested by the peer reviewers. Anything else would seem suspect unless the authors are transparent and explain to the editor and reviewers why the additions were made. COPE have flowchart guidance on how to handle authorship changes to a manuscript see here: All Flowcharts | COPE: Committee on Publication Ethics if an Editor sees a lot of late requests to add authors it may indicate authorship for sale. 

Is it needed or is there any training system/resource for reviewers to enhance the capacity to detect manuscripts from paper mills?

Yes, there is a need for more training for editors, peer reviewers and everybody else involved in manuscript processing to understand paper mills better.

Do you think depositing raw data with the statistical source code should be made mandatory? Otherwise there is no way of understanding whether the data exists at all.

Adopting Open Research initiatives such as mandatory data sharing can go some way to help tackle the problem. However, the data itself can still be manipulated.

What about review papers or manuscripts with a format that do not include any "data"?

It would still be possible for commentary or review-style papers to originate from a papermill.

Does "the third party" include medical writers as well? Some authors ask medical writers to submit their articles to the online system on behalf of them in addition to writing the papers.

It is best practice for authors to submit their manuscripts themselves and for any assistance from medical writers should be declared in the acknowledgements of the paper.

The COPE/STM report recommends "reviewing" retraction processes. What specifically do you think should we be looking at changing?

COPE’s retraction guidelines give advice and guidance for editors Retraction guidelines | COPE: Committee on Publication Ethics. However, if a number of papermill articles have to be retracted it can be time consuming to follow up with authors and institutions. If the editor/publisher has sufficient evidence, it may be more efficient to deviate slightly from COPE guidance i.e.presenting the evidence for the decision to retract rather than engaging in seeking explanations from authors.

Should there be any limit on the journal editors over number of submissions to the particular journals in which they are on the editorial board? Will this solve the increasing malpractice or misuse of their role?

The problem of manipulation of the publishing process does not lie with bonafide editors, but rather with bad actors or guest editors who deviate from expected peer review and publishing norms in order to get their manuscripts, or those from others, published.

Regarding the tools for the integrity hub, will the tools identify red flags only? Can you discuss how the human decision making oversight will be included?

It will be a combination of using tools to inform human decision making and editorial oversight.

If authors are using external (outsourced) services for manuscript prep, and are declaring (acknowledging) names of those who have assisted (perhaps with payments), then does this question their integrity?

Not if these are bonafide services and they have been declared in the acknowledgements. Please see the European Medical Writers Association guidelines for more information.

Why are guest editors being highlighted as particular targets for concern rather than regular editors?

While editors who are members of a journal’s editorial board may also not act in accordance with best practice guidelines it seems that guest editors and Special Issues are particularly vulnerable to manipulation.

Listening to the presenters and contributors, the whole system seems to be geared to paper mill success - overworked staff, insufficient reviewers with sufficient time to properly review, the pressure to publish limited risks if a paper is identified - this requires fundamental systemic change - what do presenters think?

Whereas previously publishers have operated on a system of ‘trust’ we are now seeing the need to operate a system of ‘verification’ and a need for tools and technology to support the publishing system. However, while that will bring about improvements in publishing, it won’t address the root cause of the issues.

How trustworthy is iThenticate?

 iThenticate is a tool to detect similarity and like all tools it can be evaded, for example, by authors changing key words or phrases to avoid detection. Also, indicators of text similarity is not diagnostic of plagiarism itself. This requires interpretation of the report, where the overlap appears and the sources involved.

Retraction will not affect the validity of citations now. If the products of the papermill are proven to be manipulating citations, should these citations be removal?

If a published article has extensive citation manipulation then it may need to be retracted.

Many submissions have multiple authors. If one author has cheated in any way, do you tar all the co-authors with the same brush?

Retraction notices can often make it clear if authors agree or not with the retraction and if particular authors are responsible. 

What about behaviour that suggests collusion or conflicts of interest between editors and peer reviewers? Does that also count as peer review manipulation?

Any behaviour that prevents or undermines the independent assessment of a manuscript by a peer is bad practice and can be intentional peer review manipulation (as it says in the definition you noted in your first question). Peer review manipulation is not restricted to cases wherein there are concerns about fake email address, misrepresentation of identity, or falsified review comments. Instead, this is inclusive of any efforts to manipulate peer review or its outcome for a given manuscript. It is hard to comment on the scenario described because the information provided is limited.  If there was a consistent pattern of the same editor and peer reviewers repeatedly inviting each other to peer review their manuscripts, this would be highly suspicious and warrant further investigation. This could include situations wherein the editor and/or reviewers have competing interests, as well as situations wherein editors or reviewers are working as part of a paper mill enterprise.

Back to top