‘That’s my research you’re citing’: When your research leaves home and finds new lodgings

A message popped up on my LinkedIn feed on 18 October at 2.48pm, suggesting I watch Parliamentary question time. Something of interest to me would be mentioned.

After a bit of navigating the New Zealand Parliament website, sure enough, I find a stream of the day’s coverage from the House so far, and there’s health minister Andrew Little quoting a 2019 paper on which I was a co-author along with many University of Auckland colleagues.

The subject? The “Shorter Stays” target for hospital emergency departments in the early 2010s and why and how the funders, the DHBs, gamed that target. The paper appeared in the International Journal of Health Policy Management. ¹

Surely it is every researcher’s dream to have their work splashed about in Parliament – a sign that years of unseen labour analysing data, drafting research manuscripts, responding to voluminous comments by anonymous reviewers – has finally borne fruit and had an impact on public policy.

A minute or so later, I find the National Party’s health spokesperson Shane Reti is quoting different research, from the New Zealand Medical Journal, showing the ED target saved 700 lives in one year alone.²

Hang on, that sounded familiar – the paper he quotes came from the same research team and project. I wasn’t part of the author team for the second paper, but part of me still felt a small warm inner glow that “our” research was making even more of an impact. But wait, our research is being used to counter claims that were supported by our research. How could that be? Were our findings that inconsistent?

Then the policy studies part of my brain kicks in – the part that has absolutely no illusions about how research gets used in policy.

Of course, our research was being weaponised in partisan political debate about the state of emergency departments today, who’s to blame, and whether the shorter stays in ED target should be resurrected.

According to Mr Little, the first article “was an excoriating report about how those targets had been implemented”. He said the research found that National’s policy resulted in gaming behaviour in the form of clock-stopping and decanting patients to short-stay units or observation beds to avoid target breaches. Well, we never referred to the National Party by name. We also said in that article that targets were gamed because there was opportunity to game them.

So one possible response might be to remove the opportunity to falsify information, by investing more in an independent way of monitoring ED length of stay. I’m not sure that this constitutes “excoriating”, although I guess our intention was to “peel back the skin” and understand exactly what went on in the implementation of the ED target.

Eight days later, the gaming article featured on Newshub, which reported that Mr Little’s press secretary had emailed a copy of the paper as evidence that “targets don’t work”.

But, according to both Dr Reti in Parliament and the Newshub item, the Australasian College of Emergency Medicine is also calling for the ED target to make an encore.

An interesting finding from our research into ED target implementation was that between 2009 and 2012, an estimated $50 million of DHB expenditure was devoted to ED target implementation. This was at a time when health spending per capita did not rise at all. DHBs could only have found the money by paring back expenditure in other (presumably vital) areas.

We will never know which services suffered – although from the interim report of the Health and Disability System Review panel, we know primary and community-based health services bore some of the brunt. This should remind us, however, that the key reason the college has supported use of targets is that they have proven the most effective way available of leveraging resources when resources are scarce.

The ED debate is a classic example of what Carol Weiss, way back in the 1970s, referred to as the “political/tactical use of research in policy”, and no researcher is immune from this happening to their work.

It all goes to show that the process of research informing policy doesn’t take place in a political vacuum. Policy decisions typically create winners and losers, and political parties and interest groups mobilise in order to increase their chance of winning or not losing.

As researchers, we don’t get to control whether and how our research is used. To argue that research can and should be quarantined from these political pressures is naive and unrealistic. If we want elected politicians to take notice of research, this is the price to be paid.

One positive to take from this highly partisan use of the Shorter Stays in ED research is that if you put Mr Little’s and Dr Reti’s arguments together, you actually end up with a pretty good summary of the key messages of our overall research on ED target implementation.

Yes, the target had an immediate impact in improving ED conditions and saving lives. However, over time, the effectiveness waned as there were limits to the process improvements within ED that could be made. After 18 months, the gaming behaviour really kicked in once managers and clinicians had no remaining levers to pull.

Speaking for myself (and not on behalf of the research team), I seriously doubt whether a reintroduction of an ED target will result in more than a short-term fix in the context of workforce shortages and an increased burden of infectious disease, not to mention the longer-term factors driving ever-growing ED utilisation.

But if research impact is what counts, then it’s hard to go past a Canadian musical theatre offering, Larry Saves the Canadian Healthcare System. Check out episodes 2 to 4 for a great overview of ED crowding. This series is the brainchild of Canadian health services researcher, playwright and songwriter Sara Kreindler, and I suspect it may have more impact on policy than any individual research paper I will ever be part of.

Why bust a boiler to generate research-informed policy with all its unpredictability. Research-informed musical theatre seems much more satisfying!

Tim Tenbensel is associate professor, health policy, in the School of Population Health at the University of Auckland