
Christopher Snowdon is a Fellow of the Adam Smith Institute and co-author of a new report on minimum alcohol pricing. Follow Christopher on Twitter.
The concept of evidence-based policy-making is
appealing. It certainly beats prejudice-based policy-making or bribery-based
policy-making. But while the general idea is to be applauded, political
decisions can only be as good as the evidence they are based on. In the realm
of public health, where politicians are under constant pressure to legislate,
there is a growing tendency to treat speculation as science.
For example, last year it was earnestly predicted that
a ban on below-cost alcohol would “save 21 lives a year”. In a country of 62
million souls and half a million annual deaths, 21 is statistically
indistinguishable from zero, but since this peculiarly precise figure came from
a professor of public health at Sheffield University, disbelief was suspended.
If the appeal to authority was not enough, credibility was secured by the fact
that a computer model was involved.
The same computer predicts that minimum pricing for
alcohol will save 60 lives a year in Scotland. Again, this is far too low a number ever to be tested in the real world, but if you don’t like that number, you can
choose from plenty of others. The Sheffield model has variously predicted that
900, 3,393, “more than 1,000” or “nearly 10,000” lives a year will be "saved" by minimum pricing in England. It told Panorama
that the policy will save 5,000 lives a year amongst the over-65 age group
alone, although this was later retracted (the Sheffield academics blamed "human
error").
These ever-changing estimates have had a profound
effect on the debate about minimum pricing. The public has been given the
impression that the life-saving benefits of the policy have been scientifically
established. They have not. Computer models are only as good as the assumptions
they are based on and, as a new Adam Smith Institute report shows, the
Sheffield alcohol model is based on some very questionable assumptions indeed.
As a free market think tank, the Adam Smith Institute
freely admits that it is opposed to the state placing a minimum price on any
product. However, the arguments made in this report are not based on opinion,
but on an analysis of the data by John C. Duffy, a statistician with forty
years experience in the field of alcohol research. As he explains, models of
this sort are prone to the problem of "garbage in, garbage out". If you tell a
computer that minimum pricing will reduce harmful drinking and alcohol-related
mortality, it will not try to disagree with you. If you don’t ask it to
estimate how many people will buy alcohol from the black market or cut down on
their food or heating budgets to pay for more expensive booze, it will not tell
you about the unintended consequences.
It is by no means certain that putting a floor price on
alcohol will reduce alcohol consumption. Even if it does, there is no guarantee
that lower overall consumption will lead to fewer alcohol-related deaths.
Evidence from around the world shows that changes in price never produce
predictable results. The large decline in alcohol consumption seen in Britain
in the last decade has not resulted in the health outcomes that the Sheffield
model would predict. Certainly there is no reason to assume – as the Sheffield
model does – that heavy drinkers are more
likely to reduce their consumption when prices rise. A mountain of evidence
exists to confirm the common sense view that dependent drinkers are less price sensitive than casual
consumers.
Upon
such bold assumptions do the headline-grabbing predictions for minimum pricing
rest. Many of the flaws in the Sheffield model are of a technical nature, but
are no less serious for that. The truth is that minimum pricing might reduce
alcohol harm, or it might increase it, or it might bring about other unexpected
consequences, good or bad (Donald Rumsfeld’s "unknown unknowns"). There is no
shame in saying that we simply do not know, but in the era of evidence-based
policy, it seems that speculative statistics are considered superior to no
statistics at all. The aura of scientific certainty, or even mild confidence,
in computer-generated numbers based on dubious assumptions is misplaced. An
admission that the evidence base is, to all intents and purposes, non-existent
is less likely to mislead decision-makers than a spurious prediction.
Evidence-based
policy is a laudable aim. There should be more of it. But speculation about
future events is qualitatively different from falsifiable, observable evidence
in the real world. It is crucial that we make this distinction between the real
and the imagined, and we must remember that when politicians extol the virtues
of evidence-based policy, the first thing to get politicised is the evidence
base.
Christopher Snowdon is a Fellow of the Adam Smith Institute and co-author of a new report on minimum alcohol pricing. Follow Christopher on Twitter.
The concept of evidence-based policy-making is
appealing. It certainly beats prejudice-based policy-making or bribery-based
policy-making. But while the general idea is to be applauded, political
decisions can only be as good as the evidence they are based on. In the realm
of public health, where politicians are under constant pressure to legislate,
there is a growing tendency to treat speculation as science.
For example, last year it was earnestly predicted that
a ban on below-cost alcohol would “save 21 lives a year”. In a country of 62
million souls and half a million annual deaths, 21 is statistically
indistinguishable from zero, but since this peculiarly precise figure came from
a professor of public health at Sheffield University, disbelief was suspended.
If the appeal to authority was not enough, credibility was secured by the fact
that a computer model was involved.
The same computer predicts that minimum pricing for
alcohol will save 60 lives a year in Scotland. Again, this is far too low a number ever to be tested in the real world, but if you don’t like that number, you can
choose from plenty of others. The Sheffield model has variously predicted that
900, 3,393, “more than 1,000” or “nearly 10,000” lives a year will be "saved" by minimum pricing in England. It told Panorama
that the policy will save 5,000 lives a year amongst the over-65 age group
alone, although this was later retracted (the Sheffield academics blamed "human
error").
These ever-changing estimates have had a profound
effect on the debate about minimum pricing. The public has been given the
impression that the life-saving benefits of the policy have been scientifically
established. They have not. Computer models are only as good as the assumptions
they are based on and, as a new Adam Smith Institute report shows, the
Sheffield alcohol model is based on some very questionable assumptions indeed.
As a free market think tank, the Adam Smith Institute
freely admits that it is opposed to the state placing a minimum price on any
product. However, the arguments made in this report are not based on opinion,
but on an analysis of the data by John C. Duffy, a statistician with forty
years experience in the field of alcohol research. As he explains, models of
this sort are prone to the problem of "garbage in, garbage out". If you tell a
computer that minimum pricing will reduce harmful drinking and alcohol-related
mortality, it will not try to disagree with you. If you don’t ask it to
estimate how many people will buy alcohol from the black market or cut down on
their food or heating budgets to pay for more expensive booze, it will not tell
you about the unintended consequences.
It is by no means certain that putting a floor price on
alcohol will reduce alcohol consumption. Even if it does, there is no guarantee
that lower overall consumption will lead to fewer alcohol-related deaths.
Evidence from around the world shows that changes in price never produce
predictable results. The large decline in alcohol consumption seen in Britain
in the last decade has not resulted in the health outcomes that the Sheffield
model would predict. Certainly there is no reason to assume – as the Sheffield
model does – that heavy drinkers are more
likely to reduce their consumption when prices rise. A mountain of evidence
exists to confirm the common sense view that dependent drinkers are less price sensitive than casual
consumers.
Upon
such bold assumptions do the headline-grabbing predictions for minimum pricing
rest. Many of the flaws in the Sheffield model are of a technical nature, but
are no less serious for that. The truth is that minimum pricing might reduce
alcohol harm, or it might increase it, or it might bring about other unexpected
consequences, good or bad (Donald Rumsfeld’s "unknown unknowns"). There is no
shame in saying that we simply do not know, but in the era of evidence-based
policy, it seems that speculative statistics are considered superior to no
statistics at all. The aura of scientific certainty, or even mild confidence,
in computer-generated numbers based on dubious assumptions is misplaced. An
admission that the evidence base is, to all intents and purposes, non-existent
is less likely to mislead decision-makers than a spurious prediction.
Evidence-based
policy is a laudable aim. There should be more of it. But speculation about
future events is qualitatively different from falsifiable, observable evidence
in the real world. It is crucial that we make this distinction between the real
and the imagined, and we must remember that when politicians extol the virtues
of evidence-based policy, the first thing to get politicised is the evidence
base.