Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework
We are pleased that Wharam, Serumaga and Soumerai share the concerns
that we addressed in the paper. Universal programmes such as the Quality
and Outcomes Framework eliminate control groups and preclude the use of
certain study designs, and we agree that for a programme as revolutionary
as the QOF, trialling would have been preferable to universal
implementation. In the absence of control groups, however, an Interrupted
Time Series design is the best analytic approach - indeed it is the
approach Serumaga and Soumerai used themselves [1]. Attentive readers will
note that our model was log-linear rather than linear, and as such it did
in fact gradually plateau as achievement scores increased. In the pre-
intervention period this model was an excellent fit to the indicator
trends. Post-intervention, the trends for incentivized indicators
increased above those predicted by this model, whereas the trends for non-
incentivised indicators declined.
It is possible that changes in recording activity contributed to our
findings. However, the results of laboratory tests are mostly
electronically downloaded to practices' clinical computing systems, so
documentation of these activities is not affected by physician behavior.
With respect to prioritisation, every activity we selected - whether
incentivised or non-incentivised - was recommended in guidelines.
Activities were excluded from the QOF for a variety of political and
pragmatic reasons, not necessarily because they were deemed less
important, and some controversial activities were included in the scheme.
This is reflected in the pre-QOF achievement rates - for example: the four
activities with the highest achievement rates in 2002/3, which were
presumably valued by practices, were not subsequently included in the QOF.
Re-prioritisation can occur at any time, therefore if prioritisation and
incentivisation were unrelated it would be an extraordinary coincidence if
- as we found - clinicians started ranking incentivised activities as
higher priority just at the point when incentives were introduced.
[1] Serumaga B, Ross-Degnan D, Avery A, Elliott R, Majumdar S, Zhang
F, Soumerai S (2011). Has pay-for-performance improved the management and
outcomes of hypertension in the United Kingdom? An interrupted time-series
study. BMJ 2011;342:d108 doi:10.1136/bmj.d108
Competing interests:
No competing interests
09 August 2011
Tim Doran
Clinical Research Fellow
David Reeves, Evangelos Kontopantelis, Jose M Valderas, Stephen Campbell, Chris Salisbury.
Rapid Response:
A sad fact widely known
We are pleased that Wharam, Serumaga and Soumerai share the concerns
that we addressed in the paper. Universal programmes such as the Quality
and Outcomes Framework eliminate control groups and preclude the use of
certain study designs, and we agree that for a programme as revolutionary
as the QOF, trialling would have been preferable to universal
implementation. In the absence of control groups, however, an Interrupted
Time Series design is the best analytic approach - indeed it is the
approach Serumaga and Soumerai used themselves [1]. Attentive readers will
note that our model was log-linear rather than linear, and as such it did
in fact gradually plateau as achievement scores increased. In the pre-
intervention period this model was an excellent fit to the indicator
trends. Post-intervention, the trends for incentivized indicators
increased above those predicted by this model, whereas the trends for non-
incentivised indicators declined.
It is possible that changes in recording activity contributed to our
findings. However, the results of laboratory tests are mostly
electronically downloaded to practices' clinical computing systems, so
documentation of these activities is not affected by physician behavior.
With respect to prioritisation, every activity we selected - whether
incentivised or non-incentivised - was recommended in guidelines.
Activities were excluded from the QOF for a variety of political and
pragmatic reasons, not necessarily because they were deemed less
important, and some controversial activities were included in the scheme.
This is reflected in the pre-QOF achievement rates - for example: the four
activities with the highest achievement rates in 2002/3, which were
presumably valued by practices, were not subsequently included in the QOF.
Re-prioritisation can occur at any time, therefore if prioritisation and
incentivisation were unrelated it would be an extraordinary coincidence if
- as we found - clinicians started ranking incentivised activities as
higher priority just at the point when incentives were introduced.
[1] Serumaga B, Ross-Degnan D, Avery A, Elliott R, Majumdar S, Zhang
F, Soumerai S (2011). Has pay-for-performance improved the management and
outcomes of hypertension in the United Kingdom? An interrupted time-series
study. BMJ 2011;342:d108 doi:10.1136/bmj.d108
Competing interests: No competing interests