Intended for healthcare professionals

  1. Research
  2. Small study effects in...
  3. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study
CCBYNC Open access
Research

Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study

BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c3515 (Published 16 July 2010) Cite this as: BMJ 2010;341:c3515
  1. Eveline Nüesch, research fellow12,
  2. Sven Trelle, associate director12,
  3. Stephan Reichenbach, senior research fellow13,
  4. Anne W S Rutjes, senior research fellow14,
  5. Beatrice Tschannen, research fellow1,
  6. Douglas G Altman, director and professor of statistics in medicine5,
  7. Matthias Egger, head of department and professor of epidemiology and public health1,
  8. Peter Jüni, head of division and professor of clinical epidemiology12
  1. 1Institute of Social and Preventive Medicine (ISPM), University of Bern, Switzerland
  2. 2CTU Bern, Bern University Hospital, Switzerland
  3. 3Department of Rheumatology, Immunology and Allergology, Bern University Hospital, Switzerland
  4. 4Laboratory of Clinical Epidemiology of Cardiovascular Disease, Department of Clinical Pharmacology and Epidemiology, Consorzio Mario Negri Sud, Santa Maria Imbaro, Italy.
  5. 5Centre for Statistics in Medicine, University of Oxford, Oxford
  1. Correspondence to: P Jüni juni{at}ispm.unibe.ch
  • Accepted 30 April 2010

Abstract

Objective To examine the presence and extent of small study effects in clinical osteoarthritis research.

Design Meta-epidemiological study.

Data sources 13 meta-analyses including 153 randomised trials (41 605 patients) that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patients’ reported pain as an outcome.

Methods We compared estimated benefits of treatment between large trials (at least 100 patients per arm) and small trials, explored funnel plots supplemented with lines of predicted effects and contours of significance, and used three approaches to estimate treatment effects: meta-analyses including all trials irrespective of sample size, meta-analyses restricted to large trials, and treatment effects predicted for large trials.

Results On average, treatment effects were more beneficial in small than in large trials (difference in effect sizes −0.21, 95% confidence interval −0.34 to −0.08, P=0.001). Depending on criteria used, six to eight funnel plots indicated small study effects. In six of 13 meta-analyses, the overall pooled estimate suggested a clinically relevant, significant benefit of treatment, whereas analyses restricted to large trials and predicted effects in large trials yielded smaller non-significant estimates.

Conclusions Small study effects can often distort results of meta-analyses. The influence of small trials on estimated treatment effects should be routinely assessed.

Footnotes

  • We thank Sacha Blank, Elizabeth Bürgi, Liz King, Katharina Liewald, Linda Nartey, Martin Scherer, and Rebekka Sterchi for contributing to data extraction. We are grateful to Malcolm Sturdy for the development and maintenance of the database.

  • Contributors: EN and PJ conceived the study and developed the protocol. EN, ST, SR, AWSR, and BT were responsible for the acquisition of the data. EN and PJ did the analysis and interpreted the analysis in collaboration with ST, SR, AWSR, BT, DGA, and ME. EN and PJ wrote the first draft of the manuscript. All authors critically revised the manuscript for important intellectual content and approved the final version of the manuscript. PJ and SR obtained public funding. PJ provided administrative, technical, and logistic support. EN and PJ are guarantors.

  • Funding: The study was funded by the Swiss National Science Foundation (grant No 4053-40-104762/3 and 3200-066378) through grants to PJ and SR and was part of the Swiss National Science Foundation’s National Research Programme 53 on musculoskeletal health. SR’s research fellowship was funded by the Swiss National Science Foundation (grant No PBBEB-115067). DGA was supported by Cancer Research UK. PJ was a PROSPER (programme for social medicine, preventive and epidemiological research) fellow funded by the Swiss National Science Foundation (grant No 3233-066377). The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all data of the study and had final responsibility for the decision to submit for publication.

  • Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that all authors had: (1) No financial support for the submitted work from anyone other than their employer; (2) No financial relationships with commercial entities that might have an interest in the submitted work; (3) No spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; (4) No Non-financial interests that may be relevant to the submitted work.

  • Ethical approval: Not required.

  • Data sharing: Data sharing: no additional data available.

  • Accepted 30 April 2010

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

View Full Text
Back to top