Data integrity of 10 other randomized controlled trials of an author with a retracted paper
Esmee M. Bordewijk1 2, Rui Wang1, Madelon van Wely2, Wentao Li1, Ben W. Mol1
1 Department of Obstetrics and Gynecology, Monash University, Clayton, Australia
2 Centre for Reproductive Medicine, Amsterdam UMC, Amsterdam, The Netherlands
In September 2011, a study by Dr. Tarek Shokeir and colleagues was retracted from Fertility and Sterilityâ (1), as it “duplicated parts of a paper that had already appeared in Human Reproduction” (2). In fact, the authors purportedly copied parts of text, tables, and key numbers from Perez Medina et al. The Perez-Medina paper assessed whether hysteroscopic polypectomy before intrauterine insemination (IUI) improved fertility outcomes as compared to no treatment. They included 215 infertile women with endometrial polyps (EP) undergoing IUI and randomly allocated them to hysteroscopic polypectomy or a diagnostic hysteroscopy and polyp biopsy. A total of 93 pregnancies occurred, 64 in the study group and 29 in the control group. Women in the study group had a better possibility of becoming pregnant after polypectomy, with a relative risk of 2.1 (95% confidence interval 1.5–2.9). Shokeir addressed a different topic in gynaecology, namely myomas instead of polyps, and aimed to determine whether hysteroscopic myomectomy in women with otherwise unexplained infertility achieved better pregnancy rates than no intervention. They reported 215 women who had submucous myomas with unexplained primary infertility. The women would be randomly allocated to hysteroscopic myomectomy or a diagnostic hysteroscopy and myoma biopsy. A total of 93 pregnancies occurred, 64 (63.4%) in the study group and 29 (28.2%) in the control group. These numbers were identical to those of Perez-Medina. The first author Shokeir has published 10 other RCTs that by our knowledge have never been investigated. Here, we present an in-depth analysis of these 10 RCTs from Shokeir (Table 1. Articles 1-10). Among these RCTs, six were published before the above-mentioned retraction (Table 1. Articles 1-3 & 8-10) and four were published afterward (Table 1. Articles 4-7).
First, we noticed an overlap in the title of Shokeir’s retracted article with the article of Perez-Medina from which he copied parts of text, tables and key numbers.
- Submucous myomas and their implications in the pregnancy rates of patients with otherwise unexplained primary infertility undergoing hysteroscopic myomectomy: a randomized matched control study (1).
- Endometrial polyps and their implication in the pregnancy rates of patients undergoing intrauterine insemination: a prospective, randomized study (2).
In view of this pattern of similarity in the titles, text, and tables, we searched PubMed for similarities in titles between Shokeir’s RCTs and earlier published studies and subsequently evaluated whether there were apparent overlaps in baseline and/or outcome tables. For two RCTs on which Shokeir is first author, we noticed similarities in titles with earlier studies from other research groups. The title of Shokeir 2011 “An RCT: use of oxytocin drip during hysteroscopic endometrial resection and its effect on operative blood loss and glycine deficit” (Table 1. Article 1), showed an overlap with “A randomized prospective comparative study of general versus epidural anesthesia for transcervical hysteroscopic endometrial resection.” published by Goldenberg et al. in the American Journal of Obstetrics & Gynecology in 2001 (3). Article 1 of Shokeir reported on 48 women with abnormal uterine bleeding unresponsive to conservative medical management scheduled to hysteroscopic endometrial resection and randomized to either oxytocin or saline infusion. Goldenberg et al. reported on 24 women with abnormal uterine bleeding unresponsive to conservative medical management scheduled for hysteroscopic endometrial resection randomized to either general or epidural anesthesia. Article 1 of Shokier has identical mean and SD values for uterine cavity length, operating time, and p-values as Goldenberg’s study (Figure 1).
Similarly, for Shokeir 2009 “The efficacy of Implanon for the treatment of chronic pelvic pain associated with pelvic congestion: 1-year randomized controlled pilot study” (Table 1. Article 2), we found a study by Soysal et al. published in Human Reproduction in 2001 entitled “A randomized controlled trial of goserelin and medroxyprogesterone acetate in the treatment of pelvic congestion” (4). The Soysal study included 47 patients randomized to receive Goserelin or medroxyprogesterone acetate in the treatment of pelvic congestion. The baseline characteristics table of Article 2 of Shokier is 100% identical to the table of baseline characteristics in Soysal’s study (Figure 2).
When we used Google to search the Internet using the exact values of the variables age, body mass index and period of infertility in Shokeir 2006 “Tamoxifen citrate for women with unexplained infertility” (Table 1. Article 3), we found overlaps with data in the study of Fujii et al. published in Fertility and Sterilityâ in 1997 (5), which included 33 women with unexplained infertility randomized to receive CC or no ovulation-induction drugs. Numbers and standard deviations in baseline and outcomes tables between the two studies were the same or differed with factor two (Figure 3).
For the four RCTs (Table 1, Articles 4-7) published after 2011 (i.e., after the above-mentioned retraction notice in Fertility and Sterilityâ (1)), we did not find similarities in terms of title or baseline, outcome tables with those in other studies in the literature. However, when we evaluated for these studies the robustness of randomization using Monte Carlo simulations and Kolmogorov-Smirnov test (6, 7), the distribution of baseline variables significantly violated the expected uniform distribution (p-value=0.002679), which suggests a low probability that these baseline characteristics are the results of proper randomization (Figure 4, black dots).
Apart from the retracted paper, Dr. Tarek Shokeir is the first author of seven and co-author of three RCTs published between August 2006 and November 2016 in seven different journals. We herein present three RCTs of which he is the lead author reporting identical values as three studies from other research groups on relevant but different topics (2-4). Our analysis of four RCTs of which Shokeir is the first author and published after the retraction of his publication in Fertility and Sterilityâ shows that these four RCTs are unlikely to be the results of properly conducted randomization. Two other articles of which Shokeir is a co-author (Table 1. Articles 8 & 9) have similar values in baseline and outcome-tables with several other papers, for which we have expressed our concerns separately (8). We, therefore, express our concerns on nine of the 10 RCTs by Shokeir – whether they were truly executed as RCTs.
Meanwhile, these 10 published RCTs have been cited 254 times (Scopus), of which 87 times in systematic reviews. Consequently, erroneous practice recommendations might have been drawn and as a result, wrong treatments might have been offered to patients. Based on these findings, we would call for a critical investigation of all papers published by Shokeir (>30), including non-randomized studies.
We would like to make a plea for a clear, universal policy towards authors who publish papers including likely fabricated or falsified data. When a journal suspects integrity problems, it could be considered to collect the following information: original datasets, protocols, ethics approval and documentation of study medication (placebo). The Committee on Publication Ethics (COPE) guidelines (https://publicationethics.org/) do not allow authors or institutions to be black listed. They do allo, however, information sharing among Editors in Chief (EICs) regarding cases of suspected misconduct (9). EICs of involved journals are more likely to conduct investigations with greater efficiency and effectiveness while respecting confidentiality of the submissions. Authors of referring studies have the responsibility to revise involved meta-analyses, reviews, and guidelines. This might be time-consuming, but necessary for the safety of patients. Ideally, once a study with compromised integrity is retracted, authors of studies referring to the retracted paper should be made aware of this.
There are known techniques which can be applied by journals to maximize data integrity of trials. Apart from insisting on compulsory prospective trial registration and verifying the original dataset, protocol and/or documentation of study medication as mentioned above, one can use plagiarism detection software (10) and dedicated statistical methods such as leading digits analysis and calculating the probability of randomization using baseline characteristics to detect integrity issues (11, 12). Other methods may also be considered such as using the REAPPRAISED checklist before publication (13), publicizing measures taken to minimize the risk of fabricated data, sharing of the review process interactions (i.e. the reviewer's comments and the responses from the authors), and taking extra precautions in future submissions from authors with a history of retractions due to integrity issues. We would also like to plea for a more sophisticated mechanism to detect data plagiarism. More specifically, there is a desperate need for adequate software to check plagiarism of data within tables, which is a much more important issue than the textual plagiarism that is monitored by most journals nowadays. Furthermore, when RCTs are retracted because of data fabrication, other works of the authors should be investigated.
- . Retraction. Submucous myomas and their implications in the pregnancy rates of patients with otherwise unexplained primary infertility undergoing hysteroscopic myomectomy: a randomized matched control study. Fertility and sterility 2011;96:800.
- Perez-Medina T, Bajo-Arenas J, Salazar F, Redondo T, Sanfrutos L, Alvarez P et al. Endometrial polyps and their implication in the pregnancy rates of patients undergoing intrauterine insemination: a prospective, randomized study. Human reproduction (Oxford, England) 2005;20:1632-5.
- Goldenberg M, Cohen SB, Etchin A, Mashiach S, Seidman DS. A randomized prospective comparative study of general versus epidural anesthesia for transcervical hysteroscopic endometrial resection. American journal of obstetrics and gynecology 2001;184:273-6.
- Soysal ME, Soysal S, Vicdan K, Ozer S. A randomized controlled trial of goserelin and medroxyprogesterone acetate in the treatment of pelvic congestion. Human reproduction (Oxford, England) 2001;16:931-9.
- Fujii S, Fukui A, Fukushi Y, Kagiya A, Sato S, Saito Y. The effects of clomiphene citrate on normally ovulatory women. Fertility and sterility 1997;68:997-9.
- Li W, Suke S, Wertaschnigg D, Lensen S, Wang R, Gurrin L et al. Randomised controlled trials evaluating endometrial scratching: assessment of methodological issues. Human reproduction (Oxford, England) 2019;34:2372-80.
- Carlisle JB, Loadsman JA. Evidence for non-random sampling in randomised, controlled trials by Yuhji Saitoh. Anaesthesia 2017;72:17-27.
- Bordewijk EM, Wang R, Askie LM, Gurrin LC, Thornton JG, van Wely M et al. Data integrity of 35 randomised controlled trials in women’ health. European Journal of Obstetrics and Gynecology and Reproductive Biology 2020.
- COPE. Sharing of Information Among EiCs guidelines. In. https://publicationethics.org/files/Sharing%20_of_Information_Among_EiCs_guidelines_web_version_0.pdf, accessed 08-05-2020.
- iThenticate. In. http://www.ithenticate.com/, accessed 08-05-2020.
- Hullemann S, Schupfer G, Mauch J. Application of Benford's law: a valuable tool for detecting scientific papers with fabricated data? : A case study using proven falsified articles against a comparison group. Der Anaesthesist 2017;66:795-802.
- Carlisle JB, Dexter F, Pandit JJ, Shafer SL, Yentis SM. Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials. Anaesthesia 2015;70:848-58.
- Grey A, Bolland MJ, Avenell A, Klein AA, Gunsalus CK. Check for publication integrity before misconduct. Nature 2020;577:167-9.