Recently, we received a review report from PubMedCentral for the indexing application of one of our journals. Reviewers pointed out several shortcomings of particular articles below:
1. Discussions that did not thoroughly address limitations, and conclusions that were over-stated and/or not supported by the results.
2. Methods that were not described clearly and in sufficient detail to enable reproducibility.
3. Abstracts that did not provide sufficient detail to describe the content of the article.
PMC did not suggest specific actions. However, we checked the review reports for these articles and agree with PMC's comments. We are therefore considering what we should do with these articles.
Questions for COPE Council:
- Is Retraction appropriate for the scenarios given above?
- If not, should we publish a Correction or Expression of Concern for these cases? Do you have any other suggestions?
Advice on this case is from a small number of COPE Council Members. Most cases on the COPE website are presented to the COPE Forum where advice is offered by a wider group of COPE Members and COPE Council Members. Advice on individual cases is not formal COPE guidance.
The feedback received from PubMed Central should be assessed at two levels: the journal level, and the articles. At a journal level, the editors should engage in a thorough review of their peer review and editorial processes to identify where these issues arose and establish how they can be prevented in future. This may require updating of policies and processes, and/or team training if the review suggests that there have been systematic lapses.
At an article level, it is important for the editors to understand whether the issues identified concern only the articles flagged by PMC, or whether this is something more systematic. If other articles might be affected too then it would be essential for a larger-scale review to be carried out and for any corrective actions to apply to them as well. As part of this review the editors should also determine what the nature of the issues in the editorial process is: are they connected with a single editor, for example, or with special issues? Further exploration of the concerns will help to clarify what actions need to be taken.
More specifically, the response to the papers highlighted by PBM should be based on whether the flaws uncovered nullify their conclusions. ‘Conclusions unsupported by the data’, which is one of the scenarios listed in the feedback from PMC, could range from over-stating of findings to data falsification. Advice from a neutral, domain-expert third party could be useful in determining this.
In terms of actions, if the issues are confined to a failure to address limitations or adequately describe methods, then the editors may like to offer the authors the opportunity to correct or clarify their work since those scenarios affect content but not necessarily results. This outcome would also avoid penalising authors if the flaws in the paper should have been uncovered in editorial processes prior to publication. Papers which include over-stated or unsupported conclusions might need expressions of concerns or retractions to avoid misleading readers, especially if methods and conclusions are not aligned. This should be done in conjunction with the more thorough reviews noted above.
If expressions of concern are posted, they should detail not only the specific problems with the article, but also the more general process issues, how these have been handled by the journal, and any reparative steps taken.