We launched a Special Issue (SI) focusing on the application of a particular clinical protocol, with guest editors that have an extensive clinical history in applying this protocol. This specific protocol is currently used and promoted by a small subset of practitioners, with limited wider recognition. The SI concluded with a substantial number of published articles, including several case reports. Later in the year, we received correspondence from a reader citing serious concerns about the plausibility of the protocol that the SI and articles are centred around, and particularly its apparent effectiveness in treating various disorders. It was also noted in this correspondence that the authors, guest editors and majority of reviewers invited by the most active guest editor are all prominent proponents of this specific procedure. The SI has been investigated for methodological and conflict-of-interest concerns; on the latter point, while there do appear to be personal ties between some parties involved in the process and their institutional associations, no disqualifying conflicts have been identified.
For the methodological concerns, we struggled to obtain advice as it is a very niche topic; the chief editor that initially approved the SI proposal has expressed uncertainty in assessing the complaint due to a lack of expertise. Eventually, we obtained an independent assessment from an editorial board member with experience in a related protocol, who expressed similar concerns regarding the methodology to those in the initial complaint, stating that some of the published findings could be misleading in a damaging manner, and that the case reports in particular lacked scientific rigour. This advisor stated their opinion that most of the published articles should be re-reviewed post-publication or retracted. We have subsequently contacted the leading guest editor for clarification over some of the chief concerns about the protocol, and have received an extensive response, albeit one that we are unsure has adequately explained the flagged issues. There is one manuscript currently held in review authored by this guest editor that is a methods manuscript overviewing the protocol at the centre of this investigation; this manuscript has received 2 endorsements for publication and 2 recommendations for rejection from the 4 assigned reviewers. While the concerns voiced by the independent advisor seem applicable to this manuscript, we only received advice on the published articles, so do not have an assessment from them of this manuscript, which is making the decision more challenging for the chief editor, and we have been unable to establish further contact with this advisor after receiving the initial assessment.
Questions for the Forum
- With conflicting views on the quality and effectiveness of the method under investigation, is the guidance from one independent advisor sufficient to proceed with retraction of articles they flagged as problematic, should we fail to find any other advisors willing to help?
- With the validity of the protocol under question, should the decision to publish or reject the manuscript in review be determined by obtaining further reviewers, or should an executive decision be taken by the publisher to protect against potentially scientifically unsound content?
- For Special Issues centred around a non-mainstream technique, to what extent should conflict-of-interest checks look into the research/clinical background of the reviewers and their past experience with the technique in question?
The Forum suggested four possible solutions to the handling of the articles in the Special Issue. The first would be to publish well-formed disclaimers to contextualise the presentation of the material for readers. The Forum was informed that the publisher did consider commissioning one or two articles for the Special Issue which would provide an alternative viewpoint, but the independent advisor had felt that this would not be a suitable counterweight if the concerns about the existing papers were valid.
The second option is an extension of the first, and would be to issue an Expressions of Concern, which would represent a stronger editorial position, and could address the reservations expressed by the independent advisor. Third, if the publisher felt that the issues identified in fact fatally undermine the conclusions presented then this would merit a retraction instead. Finally, the editor could consider writing an editorial piece to go alongside any Expression of Concern or Retraction notice to explain the ethical difficulties and risks behind their decision.
Another approach is to consider conflicting views to be a normal part of scholarship, and one which is not unique to Special Issues. Editors may wish to consider how far they should facilitate a platform for these views, as long as they are not dangerous.
However, the case does also illustrate some of the problems seen with Special Issues, and the potential (even when proper processes are followed) for them to be used by groups with a particular agenda. As well as vetting guest editors, it would be appropriate to consider whether there are any agendas or conflicts at work when a proposal is received: for example, whether it is unscientific or a niche topic which will be lent credibility via publication.
On the question of reviewer selection: this will rely on the editor knowing the field well and being able to achieve a balance between reviewers. Editors should consider whether it is best practice to select people known to have opposing views, and how they balance assessing the scholarship against the impact of politics and wider debates. It is important to ensure that editorial decisions are informed by evidence but it was agreed that this can be a difficult balance to achieve.
The publisher decided to allow the Special Issue editor to publish an editorial, as it was considered unlikely that the Special Issue as a whole would be dissolved. However, they plan to initiate post-publication re-review of several of the articles with the most serious concerns to begin with, and use the feedback from the reviewers to formulate a strategy towards the Special Issue.