We published a randomised trial by six authors. Some years later, we received a letter from a researcher who had been looking into the trial in the context of a meta-analysis. She noted “implausibilities of serious concern”, including “a highly unusual balance in the distribution of baseline characteristics”, 95% CIs that were non-symmetrical about the effect estimate, and use of a stratification variable the value of which could not have been known in all patients if the trial was conducted in the way reported.
We asked the corresponding author to write a letter of response, which he eventually supplied a few months later. Owing to the author’s poor English and the level of statistical knowledge needed to assess the response, we sent the exchange of letters to a statistical reviewer. The reviewer said that the letter of concern was “completely correct” in everything it said and that the author’s explanation for the unusual degree of balance in the covariates was “rubbish”, and that the 95% CIs had either been “doctored” or “incompetently estimated”.
In the meantime, an exchange of to-and-fro letters between a different researcher and the same author was published in another journal, relating to a paper reporting on a subset of the same trial data. We were alerted to this by the editor of that journal.
We sent the reviewer’s remarks to the author of our trial, who then consulted two independent statisticians of his own. He soon contacted us to say that, “surprisingly and regretfully”, these statisticians agreed that there were implausibilities and inconsistencies in the data, and asked for more time to investigate more fully. During this time, the author of the letter expressed concern that we had not made the possibility of these problems know to our readership, so we published her letter.
The author has now sent us a more comprehensive response, admitting that the randomisation process was not as described, the 95% CIs were all wrong (he supplied a recalculation), and the trial report had omitted some details of the protocol necessary for understanding it properly (now supplied). Our reviewer suspects that, given his free admission of all this, the author is probably incompetent rather than fraudulent, but that the extent of the incompetence could not give us confidence in any of the data. What now?
The Forum agreed with the editor’s opinion that the author is probably incompetent rather than fraudulent and should be given the opportunity to redeem himself. It was suggested that perhaps the paper should be submitted for review again. The Forum noted that this was probably a good internal learning exercise in that the statistical errors should have been picked up when the statistical review of the data was performed by the journal. A suggestion was made for the journal to set up a “sin bin”. Some journals operate a “sin bin” or “publication review committee” where once a year papers which readers or others have expressed serious doubts about post-publication are reviewed to determine whether or not it was “a mistake” to publish the paper.
However, some members of the Forum argued for stronger action and suggested contacting the author’s institution. But most agreed that the paper should be retracted as the research may be unethical.
August 2008
We have managed to find details of whom to contact regarding informing the author's institution, and the deputy editor has written to him. We await a reply with anticipation.
May 2009
The institute has responded to say that an investigation is under way and will take another couple of months to conclude.
November 2009
Following an internal investigation by the author’s institution, a number of serious problems were encountered, including: lack of ethics approval, lack of written consent, lack of treatment-allocation concealment and an inability to verify the authenticity of the data. We therefore retracted the paper on 10 October 2009.