Designing and evaluating complex interventions to improve health care
BMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39108.379965.BE (Published 01 March 2007) Cite this as: BMJ 2007;334:455
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Campbell et al 1 emphasise the importance of preliminary work in
understanding the context of a complex intervention prior to testing it in
a randomised controlled trial. However, whilst they indicate that, for a
behavioural intervention for people with cardiovascular disease, a
sufficient understanding may be achieved by reviewing literature, we found
additional value in primary research before a definitive trial.
Preliminary qualitative findings with the SPHERE Study2 helped define
context and understand particular behavioural influences within the
proposed study population – such as patients’ beliefs regarding the
relative risks of stress and smoking, their need for help in managing
stress and practitioners’ poor motivation for providing services for non-
compliant patients. Exploration of the proposed trial setting revealed
the variation in administrative support which practices would require.
Defining various levels of operation within the intervention helped
develop a theoretical framework but also heightened awareness of
monitoring treatment fidelity3 and possible contextual change during the
definitive trial – due to national initiatives to improve coronary heart
disease management, local initiatives to promote physical activity and
smoking legislation. Our findings will be interpreted in the context of
these.
Our preliminary work also revealed the significance of cost
implications of behaviour change for patients and practitioners. Though
omitted from Campbell et al’s discussion,1 we support suggestions4 that
cost considerations should be included in the modelling stages of design
of trials of complex interventions. Analysis of cost-benefits is
essential for meaningful evaluation of health services interventions.
There are obvious financial implications for funding bodies in
decisions regarding appropriate allocation of time and money for
preliminary work prior to a definitive trial. Reviewers should consider
whether applicants have sufficient knowledge of the context of proposed
trials to ensure that outcome data will provide meaningful conclusions
relevant to other health service settings.
Inevitably, following preliminary work a definitive trial may not
proceed either because the intervention is unlikely to be cost effective
or it is obvious that it should be implemented.1 This may cause employment
difficulties for research staff whose possible longer term funding depends
on outcomes of short term work. Allocating funding on a staged basis may
be best accommodated within a research network framework, such as recently
initiated through the UKRCN5 whereby a core staff of researchers can share
knowledge of factors influencing the implementation and evaluation of
complex interventions, within the context of both primary and secondary
care. Researchers’ skills may be transferred between projects with
greater flexibility than if employed directly for individual projects:
this should facilitate achievement of meaningful outcomes of trials,
particularly of complex interventions.
References
1. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F,
Guthrie B, Lester H, Wilson P, Kinmonth AL. Designing and evaluating
complex interventions to improve health care. BMJ 2007;334:455-459
2. Corrigan M, Cupples ME, Smith SM, Byrne M, Leathem CS, Clerkin P,
Murphy AW. The contribution of qualitative research in designing a
complex intervention for secondary prevention of coronary heart disease in
two different healthcare systems BMC Health Services Research 2006;
6:90 doi:10.1186/1472-6963-6-90
3. Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Sharp
Minicucci, D., Ory, M., Ogedegbe, G., Orwig, D., Ernest, D. &
Czajkowski, S. Enhancing Treatment Fidelity in Health Behavior Change
Studies: Best Practices and Recommendations from the NIH Behavior Change
Consortium.
Health Psychology 2004;23(5):443-451.
4. Eldridge S, Spencer A, Cryer C, Parsons S, Underwood M, Feder G.
Why modelling a complex intervention is an important precursor to trial
design: lessons from studying an intervention to reduce fall-elated
injuries in older people. J Health Serv Res Policy 2005;10(3):133-142
5. http://www.ukcrn.org.uk/ accessed 2007-04-06
Competing interests:
None declared
Competing interests: No competing interests
The original publication of the MRC complex interventions framework
document1 in 2000 has inspired much creative thinking within a relatively
short period of time, and reflection upon its value and content is very
welcome.2 Campbell and colleagues propose combining what were originally
conceptualized as iterative theoretical, modeling and exploratory trial
phases 0-2 into a single pre-phase III stage of activities directed
towards understanding the problem that a complex intervention is aiming to
address as well as its context, and developing intervention and evaluation
methods. 2
This focus on context is extremely helpful. However, it is not clear
how what is being proposed would actually work in practice nor what
designs are being recommended for this reconceptualisation of research
tasks. The proposed new model may have unintended negative impacts such as
discouraging the conduct of, or diminishing the importance of, exploratory
trials. The authors state that “early randomised studies also have a
place” but it is not at all clear how decisions about their conduct are
proposed to be made, and consequently where their place really is.
Whilst it is entirely possible to optimise intervention and
evaluation methods in parallel rather than together, this course of action
risks depriving the researcher of useful information on effect size
obtained in directly replicable conditions which can be obtained in a well
-conducted exploratory trial. Small pilot studies may yield important data
on effects which are much less than or larger than anticipated. With large
numbers, strong study and intervention designs, and reliable evidence of
effect size, exploratory trials may also vitiate the need for later
definitive trials.
The titular use of complex interventions to improve “health care”, 2
rather than “health” as it was originally,1 as well as the illustrative
examples chosen, suggest a health services research (HSR) orientation to
large-scale multi-level interventions. Perhaps in HSR exploratory trials
are less important than in individual behavioural research, for example,
through greater use of quasi-experimental designs? HSR interventions are
indeed complex, and careful studies for their evaluation which include
appropriate levels of attention to processes and mechanisms of effects are
not straightforward to do well.
However, it would be a real cause for concern if small scale
experimentation was inadvertently discouraged, and particularly so in
relation to individual behaviour change. The development and evaluation of
behavioural interventions really do benefit from data obtained in
exploratory trials. Given the prominence of these interventions in the re-
orientation of the NHS heralded by Choosing Health, 3 surely we need more
rather than fewer trials which contribute high-quality evidence? This
White Paper clearly views much existing public health research activity as
dysfunctional, particularly in relation to timeliness. Specific
encouragement to undertake exploratory trials may help researchers and
practitioners together to make decisions about definitive trials, as well
as about what to do in practice whilst awaiting their findings.
Behavioural and organisational interventions both seem clearly to
benefit from application of this complex interventions perspective. It is
not yet clear whether there are also other types of interventions which it
is helpful to conceptualise as being complex. We should also not assume
that all behavioural interventions, for example, need to be studied as
being inherently complex. There are also interventions which can be
straightforwardly disseminated to generate substantial aggregate health
gain. In some circumstances, it may well be more important to know that
there are small effects reliably detected in large trials, than to know
precisely what are the mechanisms of these effects. Better appreciation of
the complex interventions perspective will be served by further
consideration of where (and why) it does not apply.
Jim McCambridge & Chris Bonell
LSHTM
1. Medical Research Council. A framework for development and
evaluation of RCTs for complex interventions to improve health. London:
MRC Health Services & Public Health Research Board 2000.
2. Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A.,
Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L.
Designing and evaluating complex interventions to improve health care. BMJ
2007; 334: 455-459.
3. Department of Health. Choosing Health: Making healthy choices easier.
London: Department of Health; 2004.
Competing interests:
None declared
Competing interests: No competing interests
Dear Sir
Campbell et al’s paper on designing and evaluating complex
interventions is a welcome analysis but fails to highlight the importance
of patient preference(Campbell et al. 2007).
One of the limitations of complex intervention trials is that of bias
arising from patient preference. Patient involvement is more active than
in a single drug trial involving the passive acceptance of drug A or drug
B, and this involvement has the potential to influence the outcome of the
trial depending on the degree of patient participation in the
intervention. Clearly it is not possible to blind patients in trials of
interventions requiring active participation, and this may introduce bias
in the estimate of the effectiveness of the intervention. Patients with
strong preference may refuse randomisation because of the risk of not
receiving their treatment of choice. If many eligible patients refused
randomisation, the results may not be generalisable and would have the
potential to weaken the external validity of a trial (Bower et al. 2005).
Brewin and Bradley have proposed the use of a ‘comprehensive cohort
design’ to evaluate interventions that involve active patient
participation (Brewin & Bradley 1989) In fact, the Medical Research
Council framework suggested the use of a preference trial design when
patients express a strong preference for an intervention in a complex
intervention trial (Medical Research Council 2000).
We have recently reported a randomised trial of a complex
intervention involving home based and hospital based cardiac
rehabilitation programmes after myocardial infarction. Using a patient
preference design in the study we were able to include an additional 126
(55%) patients which improved the external validity of our findings(Dalal
et al. 2006). Patient preference designs should be considered in trials of
complex interventions as they could improve recruitment and make the
results more generalisable.
References
Bower, P., King, M., Nazareth, I., Lampe, F., & Sibbald, B. 2005,
"Patient preferences in randomised controlled trials: conceptual framework
and implications for research", Soc Sci Med, vol. 61, no. 3, pp. 685-695.
Brewin, C. R. & Bradley, C. 1989, "Patient preferences and randomised
clinical trials", BMJ, vol. 299, no. 6694, pp. 313-315.
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A.,
Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L.
2007, "Designing and evaluating complex interventions to improve health
care", BMJ, vol. 334, no. 7591, pp. 455-459.
Dalal, H. M., Evans, P. H., Campbell, J. L., Taylor, R. S., Watt, A.,
Read, K. L., Mourant, A. J., Wingham, J., Thompson, D. R., & Gray, D.
J. 2006, "Home-based versus hospital-based rehabilitation after myocardial
infarction: A randomized trial with preference arms - Cornwall Heart
Attack Rehabilitation Management Study (CHARMS)", Int.J Cardiol.
Medical Research Council 2000, A framework for development and evaluation
of Randomised Control Trials for complex interventions to improve health.
Competing interests:
None declared
Competing interests: No competing interests
Confusing the concepts of complicated and complex
We suggest that the authors1 are confusing the concepts of complicated and complex in both the nature of the intervention and the nature of the context and landscape of the intervention.2
According to Glouberman and Zimmerman3 systems can be understood as being simple, complicated, complex, or chaotic. Simple and complicated systems or processes are related to separate entities or discrete activities. In contrast complex systems are based on relationships, and their properties of self-organisation, interconnectedness and evolution. Research into complex systems demonstrates that they cannot be understood solely by simple or complicated approaches to evidence, policy, planning and management.4
Simple problems, such as following a protocol, may encompass some basic issues of technique and terminology, but once these are mastered, following the "recipe" carries with it a very high assurance of success.3
Complicated problems contain subsets of simple problems but are not merely reducible to them. Their complicated nature is often related not only to the scale of a problem like open heart surgery, but also to issues of coordination or specialised expertise. Complicated problems, although their solutions are generalisable, are not simply an assembly of simple components.3
Complex problems can encompass both complicated and simple subsidiary problems, but are not reducible to either, as they too have special requirements, including an understanding of unique local conditions and their historical pathways. 3 Adaptive, self-organising social networks produce observable patterns in response to interventions, that are neither predictable nor generalisable, yet understanding them in retrospect can inform future possibilities.2
Approaches to understanding complex systems developed by Kurtz and Snowden5 for IBM international e-business management have been successfully applied with frontline health care providers.6 They categorise activities on four levels of knowledge and organisation--described as the known, the knowable, the complex and the chaotic. We argue that each is governed by a particular evidence and decision-making mode: the known (analytical/reductionist evidence-based care); the knowable (potentially ascertainable by application of evidence-based methods) and the complex (non-predictable, but potentially understandable by pattern observation) knowledge domains. The known and knowable refer mainly to simple and complicated knowledge, while complex (and chaotic) knowledge is based on understanding dynamic system patterns in which the whole is greater than the sum of the known and knowable parts.
Thus, methodology described by Campbell et al as complex intervention(s) in complex health systems should really be described as interventions and RCTs based upon the assumptions of complicated interventions in known or knowable environments. The purpose of RCTs is to compare two or more options and control for all differences, such that in future one can predict which option to implement. The approach described seeks to reduce what is complex to a complicated series of steps and processes in order to ensure greater replicability and generalisability which is laudable. Such approaches are a considerable improvement on the black box RCT of the past, yet it is misleading to use the terminology of complexity. Arguably there have been sufficient international journal articles on the subject of complexity for the meaning to have become common7.
1. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, et al. Designing and evaluating complex interventions to improve health care. British Medical Journal 2007;334(7591):455-459.
2. Martin C, Sturmberg J. General Practice - chaos, complexity and innovation. Medical Journal of Australia 2005;183(2):106-109.
3. Glouberman S, Zimmerman B. Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like? Ottawa: Discussion paper No 8. Commission on the Future of Health Care in Canada, 2002.
4. Bar-Yam Y. System Care: Multiscale analysis of Medical Errors - Eliminating errors and improving organizational capabilities. NECSI Technical Report 2004-09-01, New England Complex Systems Institute 2004:http://necsi.org/projects/yaneer/NECSITechnicalReport2004-09.pdf (accessed 12-03-05).
5. Kurtz C, Snowden D. The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal 2003;42(3):462-483.
6. Hoff T. The power of frontline workers in transforming organizations. The Upstate New York Veteran’s Health Administration: IBM Center for Healthcare Management, 2003: Available at: http://www-935.ibm.com/services/us/gbs/bus/pdf/ibm_healthcaremanagement_... (accessed Jul 2007).
7. Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care. BMJ. 2001 Sep 15;323(7313):625-8
Competing interests:
None declared
Competing interests: No competing interests