Of targets, measures and the tricks to improving health system performance

The health targets first emerged in 2007 from the then Labour Government, specifying priority areas for the ministry and the sector.

In 2009, National Party health minister Tony Ryall repurposed them into a health version of Helen Clark’s “pledge card” – a set of tangible, verifiable commitments that could be ticked off during subsequent election campaigns.

Health target regimes always attract controversy, here and abroad.

Defenders assert that they provide a clear focus, encourage creativity and innovation, and enhance the accountability of publicly funded health services. Detractors counter that targets foster tunnel vision and “gaming”, and distort priorities by siphoning resources away from important but untargeted services.

From 2009, New Zealand’s target regime provoked fear and reverence from its subjects, as DHB (and sometimes PHO) chief executives genuinely feared the phone call from the minister (and public shaming from their local news outlet) if their target performance was deemed unsatisfactory.

It now turns out that both the defenders and the detractors of targets were right.

The regime produced some genuine improvements, particularly in its early days. Esther Willing’s PhD research showed that the child immunisation target was highly successful in stimulating creative problem-solving and learning across DHBs, PHOs and primary care practices.

However, the balance between benevolent and malevolent consequences differed substantially across the six targets. Other targets such as the cancer waiting times, elective surgery numbers, and cardiovascular risk assessment were much more prone to gaming and myopia.

Research I have been involved in, funded by the Health Research Council, examined emergency department target implementation.

We found that in the first 18 months, there were genuine reductions in ED length of stay, which in all likelihood saved hundreds of lives.

These improvements were facilitated by modifications of ED and hospital systems and processes.

However, after 2011, continued achievement of the target was mainly attributable to the increasing availability of ED short-stay facilities, such as acute observation units.

Gaming behaviours such as “clock-stopping” and shifting patients “off target” also became more prevalent from 2011. As the target regime bedded down, DHBs and PHOs increasingly engaged in behaviours that hit the target but missed the point. The regime, however, was satisfied as long as the numbers looked good.

DHBs and PHOs increasingly engaged in behaviours that hit the target but missed the point

Our health targets predominantly measured outputs (activities) and processes, rather than health outcomes. Advocates of targets argue that we should hold organisations accountable for things they can control (ie, outputs and processes), rather than things subject to a much broader set of influences, such as health outcomes.

Since 2009, the fashions of public policy and management in health have changed considerably to emphasise collaboration between organisations and joint responsibility for outcomes.

Reflecting this trend, the new monarch of performance improvement, the System Level Measures framework quietly usurped the throne in 2016.

This monarch has quite a different style of governing. The idea is to use health outcomes as the centrepiece of a performance system.

Rather than emphasising accountability for improvement in these outcomes, however, the emphasis is on using these indicators to stimulate a quality improvement approach applied to local health districts as a whole. For example, by examining a district’s ambulatory-sensitive hospitalisation rate for children aged under five, you can drill down and see that some conditions (eg, skin infections, respiratory infections, dental caries) are the largest contributors. This can stimulate conversations about improving respiratory, eczema or dental services across primary and secondary care.

The approach requires starting with the outcome, and working backwards to choose appropriate activities and processes.

The onus is on collaboration between DHBs, PHOs and other non-government health providers. Indeed, the intention is that it will stimulate collaboration and give district alliances something very practical to focus on.

Implementation depends on goodwill between organisations, good collaborative processes, good data and the capacity to analyse the data. These characteristics vary enormously between New Zealand districts. Where these capacities are in short supply, the virtuous circle of quality improvement in these measures may never get started.

But can System Level Measures be harnessed to enhance accountability of health sector organisations? Some policy insiders speak of them as potentially “squaring the circle” between quality improvement and performance management. Others, including prominent international public management researchers, regard these two imperatives as largely incompatible.

Much depends on the specific indicator. The child immunisation target successfully stimulated the virtuous circle of accountability and learning. But this might be an exceptional case, in which it is easy to join the dots between activities (immunising children), processes and outcomes (increased herd immunity).

In the case both of targets and System Level Measures, the difficulty in joining these dots is the major impediment to using performance measurement to improve systems.

After a decade looking into the question, I have come to the view that relying wholesale on health targets to improve performance is analagous to prescribing steroids in treating chronic health conditions. There can be dramatic short-term improvements, but prolonged usage distorts and compromises the operation of the system.

The newer measures are more like a therapy designed to enhance the health sector body’s own healing powers – but they are only in very early stages of the experimental phase.

Health minister David Clark has signalled a revamp of health-sector performance management. Will he let (some version of) the System Level Measures mature into adulthood? Or will their life be cut off in early adolescence, for the next generation of House of Targets to take the throne?

Tim Tenbensel is head of the health systems group at the University of Auckland