Beware the noxious weed that is ‘evidence-based policy’ running rife

Fundamentally, I have only two problems with use of the term “evidence-based policy”. The first problem is the word “based”, and the second is the word “evidence”

But once the 2018 Budget is delivered, and we get a sense of the Government’s spending priorities, one thing is for sure – we will be hearing many calls that funding and policy decisions should be evidence based.

The term “evidence-based policy”, however, is something of a noxious weed that has spread from its original habitat of evidence-based medicine – where it has a useful place in the ecosystem of ideas. However, in the public policy context, the term “evidence-based” has run rife and replaced older and more suitable terms.

Fundamentally, I have only two problems with use of the term “evidence-based policy”. The first problem is the word “based”, and the second is the word “evidence”.

When anyone says that policy should be “evidence-based”, the strong inference is that influences on policy that are not related to research evidence – such as values – have no place. This exemplifies a technocratic approach to public policy. The technocratic ideal of rule by experts, which some would argue goes back as far as Plato, is fundamentally anti-democratic. It usually stems from a wish to stamp out the messy, argumentative, emotive and “irrational” features of a policy process. Those favouring a technocratic approach seek to take the politics out of policy. However, the words “policy” and “politics” have exactly the same Greek root of “polis” which means “the public”.

In essence, both politics and policy is about how we, as a public, find ways to move forward collectively when we disagree. Policy without politics is oxymoronic.

The second problem is the word “evidence”. Typically, the term evidence is used here as a synonym for peer-reviewed, published research. But the words “evidence” and “research” have different meanings.

To illustrate this, consider another context in which the term evidence is central – the setting of a legal trial. Eyewitness reports, telephone records, CCTV footage are all commonly admitted as evidence in a murder trial – but none of these require research published in peer-reviewed journals. The critical factor that distinguishes evidence from non-evidence is relevance to a central question – is Mr Smith guilty? In a trial, all participants know and agree what that question is.

In evidence-based medicine, the central question usually takes the form of “is treatment ‘A’ safer/more effective than no treatment/treatment ‘B’?” Here, the source of evidence to answer these types of questions is restricted to disciplined scientific and clinical research published in peer-reviewed journals – that’s where the conflation between evidence and research happens.

Using “evidence-based” in the context of public policy makes an unwarranted assumption that all policy can be understood in terms of definable interventions that are potentially underpinned by published research. In reality, much policy is (and should be) based on pragmatic experimentation. But an even bigger, and much more problematic assumption is that all participants in a policy issue are addressing the same question. This is rarely the case.

A celebrated example was documented a decade ago by Mark Monaghan, who studied the UK debate around the classification of cannabis. In the mid-2000s, a committee led by David Nutt (at the time the chief scientific advisor to the UK government), had the task of answering whether criminal penalties for possession and use of cannabis were commensurate with its harm. Professor Nutt’s committee recommended changing the classification from Class B to a more lenient Class C. The decision was criticised by those wanting to legalise cannabis, and by conservative lobby groups arguing for no change. What was striking was that all three sides in the debate used the bully pulpit of evidence-based policy to support their preferred stance. They were able to do so, because each was strategically asking a completely different question. Anti-prohibitionists asked “does criminalising cannabis deter people from using it?” Conservatives asked “is cannabis more harmful now than when it was first classified as a Class B drug?”

The moral of the story is that governments and experts are usually not in control of the questions that circulate in policy debate. As such, the phrase “evidence-based policy” can be used rhetorically for a range of purposes. For example, it provides alcohol and food industry groups with a very handy stick to beat public health groups. When public health advocates argue for novel interventions, their opponents will cry that such initiatives are not evidence-based.

Of course, if governments had to wait for published evidence before making policy initiatives in these areas, very little would happen.

What we can and should strive for is “research-informed policy”. It sounds less emphatic, but it is a much more reasonable and achievable imperative. n

Tim Tenbensel is head of the health systems group at the University of Auckland