Impact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study
BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g4145 (Published 01 July 2014) Cite this as: BMJ 2014;349:g4145
- Sally Hopewell, senior research fellow12,
- Gary S Collins, senior research fellow1,
- Isabelle Boutron, professor2,
- Ly-Mee Yu, senior medical statistician1,
- Jonathan Cook, associate professor13,
- Milensu Shanyinde, medical statistician1,
- Rose Wharton, medical statistician1,
- Larissa Shamseer, PhD student4,
- Douglas G Altman, director1
- 1Centre for Statistics in Medicine, University of Oxford, UK
- 2Centre d’Epidémiologie Clinique, Université Paris Descartes, INSERM U1153, France
- 3Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, UK
- 4Ottawa Hospital Research Institute, Canada
- Correspondence to: S Hopewell Centre for Statistics in Medicine, University of Oxford, Oxford OX3 7LD, UK sally.hopewell{at}csm.ox.ac.uk
- Accepted 12 June 2014
Abstract
Objective To investigate the effectiveness of open peer review as a mechanism to improve the reporting of randomised trials published in biomedical journals.
Design Retrospective before and after study.
Setting BioMed Central series medical journals.
Sample 93 primary reports of randomised trials published in BMC-series medical journals in 2012.
Main outcome measures Changes to the reporting of methodological aspects of randomised trials in manuscripts after peer review, based on the CONSORT checklist, corresponding peer reviewer reports, the type of changes requested, and the extent to which authors adhered to these requests.
Results Of the 93 trial reports, 38% (n=35) did not describe the method of random sequence generation, 54% (n=50) concealment of allocation sequence, 50% (n=46) whether the study was blinded, 34% (n=32) the sample size calculation, 35% (n=33) specification of primary and secondary outcomes, 55% (n=51) results for the primary outcome, and 90% (n=84) details of the trial protocol. The number of changes between manuscript versions was relatively small; most involved adding new information or altering existing information. Most changes requested by peer reviewers had a positive impact on the reporting of the final manuscript—for example, adding or clarifying randomisation and blinding (n=27), sample size (n=15), primary and secondary outcomes (n=16), results for primary or secondary outcomes (n=14), and toning down conclusions to reflect the results (n=27). Some changes requested by peer reviewers, however, had a negative impact, such as adding additional unplanned analyses (n=15).
Conclusion Peer reviewers fail to detect important deficiencies in reporting of the methods and results of randomised trials. The number of these changes requested by peer reviewers was relatively small. Although most had a positive impact, some were inappropriate and could have a negative impact on reporting in the final publication.
Footnotes
Contributors: SH and DA were involved in the design, implementation, and analysis of the study, and in writing the final manuscript. GC, LY, JC, MS, RW, IB, and LS were involved in the implementation of the study and in commenting on drafts of the final manuscript. SH is responsible for the overall content as guarantor.
Funding: This study received no external funding.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Data sharing: No additional data available.
Transparency: The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.