You are here

Case study: repeated complaints about meta-analysis

Case study

Journal policies and procedures

This type of case would be categorised under three COPE Core Practices:

  • Complaints and Appeals: “Journals should have a clearly described process for handling complaints against the journal, its staff, editorial board or publisher.”
  • Journal Management: “A well-described and implemented infrastructure is essential, including the business model, policies, processes and software for efficient running of an editorially independent journal, as well as the efficient management and training of editorial boards and editorial and publishing staff.”
  • Post-publication Discussions and Corrections: “Journals must allow debate post publication either on their site, through letters to the editor, or on an external moderated site, such as PubPeer. They must have mechanisms for correcting, revising or retracting articles after publication.”

The three core practices are related in this case, because the journal owner should ensure there are clear policies, procedures, guidelines, and internal processes/training on how to efficiently and fairly manage complaints and post-publication correspondence. Journals also need to appropriately handle and distinguish between informal messages (emails, online posts) and formal correspondence (submitted letters, commentaries), and between differences of opinion and when to conduct post-publication investigations. Guidelines should remind readers that all correspondence must remain civil, legal, and ethical, and that the editor’s decision is final.

The journal’s duty is to maintain the quality of its published content and the journal should follow due process and keep appropriate records. At the outset, there should be a contact person and triage system in place to identify and follow up valid concerns about content quality raised in both formal and informal correspondence received. Relevant COPE guidance documents for handling criticisms of published articles when raised by informal correspondence are “How to respond to whistleblowers when concerns are raised directly” and “How to respond to whistleblowers when concerns are raised via social media”.

Ideally, the journal would have a standard operating procedure to handle different types of concern and determine if the points raised are, for example, differences in interpretation or opinion, possible errors needing correction/clarification, allegations of misconduct, or a complaint about journal procedure. If needed, a post-publication round of peer review could be initiated. If an allegation of misconduct has been made, journal staff can follow relevant COPE flowcharts.

However, there is also a need early on to filter out frivolous, baseless, incomprehensible, or overly vague claims and old/resolved issues, as well as potential or likely cases of libel, trolling, cyberbullying, and personal attacks. Repeated and frequent correspondence from a person who disagrees with an editorial decision could become harassment and be a drain on resources. COPE has published relevant advice in “Addressing ethics complaints from complainants who submit multiple issues: COPE Discussion Document”. If necessary, the editor may need to seek advice from ethics, legal, and institutional representatives, especially if the reader persistently includes libellous, offensive, hostile, or illegal content in their correspondence.  

Correspondence

Some journals consider various factors for formal reader correspondence, such as recency, originality, length, number of illustrations and references, the need for evidence to support points, whether new data are allowed, research ethics and conflicts of interest, if peer review will be used, and a limit on the number of rounds of responses. Some journals explicitly say they do not allow the formal correspondence section to be used to report possible errors. Reader correspondence relating to post-publication corrections should be investigated separately rather than published. If appropriate and with consent, the reader may be acknowledged in the correction.

The journal should address all valid concerns from a reader, although it would be more convenient to deal with all concerns and subsequent corrections at once. The journal and authors would also need to discuss if so-called errors really are errors. Distinctions should be made between fixable mistakes in a paper; alternative viewpoints; and ongoing scholarly progress and debate where previous findings are reinterpreted or overturned by new research thanks to technological or methodological advancements. Furthermore, data or findings from new research should not normally be added post-publication as a correction to update a paper; a new research paper should be prepared. Nevertheless, some journals do allow authors to publish meta-analysis update articles and online continually updated (“living”) systematic reviews.

Transparency of meta-analyses

Meta-analyses are notoriously challenging, involving complex methods and adjustments to minimise or correct for biases. What might be perceived as an error may be the omission of an optional methodological adjustment or sub-analysis that may not affect the overall conclusion. An addendum may then be appropriate. Different sets of authors with good reason may even publish alternative meta-analyses of the same source data, as long as the differing methods are clear and justified and the previous article is cited properly. To increase robustness, transparency, and reproducibility, some journals recommend meta-analysis protocols to be standardised (eg, using PRISMA-Protocol guidelines), registered publicly in advance (eg, on PROSPERO), or submitted to a peer-reviewed journal as a protocol paper or registered report before the actual research starts.

As mentioned, for full transparency journals should clearly display their codes of practice, editorial and review policies, and complaints/appeals procedures. Such information can be based on COPE’s Core Practices and website resources, as well as other industry/society guidelines, publisher guidelines, and local regulations. Retraction is warranted only for certain reasons listed in the COPE Retraction Guidelines, such as if the conclusion is unreliable and/or there is an ethical problem with the data, research, reporting, or peer review. As part of an investigation, the editor might consider asking for the advice of the original peer reviewers of the paper, to inform the editor’s decision. The reader should be explicitly told the journal’s policies and that the editor’s decision is final. If the reader appeals and the journal is a COPE member, COPE’s Facilitation and Integrity Subcommittee could be asked for advice on whether due process had been followed.

A possible positive outcome could be a future collaboration between the parties to improve and build on the research. Unfortunately, if the parties are academic rivals, there is a risk that the reader’s direct correspondence could become unreasonable, personal, or vexatious.

Still, the editor could advise that the authors seek assistance from their institution, which could contact the reader’s institution if needed. The editor could also be prepared to explain the interactions and actions taken so far, if the institution later contacted the journal about the reader. If the reader’s correspondence with the institution were professional and raised a valid concern that the institution in turn referred to the journal, the editor could follow relevant COPE guidance contained in “Cooperation between research institutions and journals on research integrity cases: guidance from the Committee on Publication Ethics (COPE)”. 

Trevor Lane on behalf of the COPE Education Subcommittee

Related resource

How to respond to whistleblowers when concerns are raised directly flowchart

How to respond to whistleblowers when concerns are raised via social media flowchart

Addressing ethics complaints from complainants who submit multiple issues discussion document

Retraction Guidelines guidelines

Cooperation between research institutions and journals on research integrity cases guidelines