Gary Bennett is CEO of The Stats People and is a statistical consultant to the survey research industry.
No wonder the British Public are suspicious of experts. Hardly a day goes by without another national media comment piece about how Brexit has divided the country, supported by colourful charts and scientific-looking analysis.
The Brexiteers, we are told, are lumpen, ageing, poorly educated, UKIP supporting, socially excluded provincials; the “left behind” as they have been tagged. The Remainers are young, optimistic, educated, supporters of progressive politics, living in London or metropolitan areas. The land’s ignorant failures and never-rans dragging the rest of the country screaming into the abyss is a compelling image.
The closeness of the Brexit vote share (Leavers falling just short of 52 per cent) is cited as evidence of a weak mandate for leaving the EU. Those supporting this narrative of the stupid versus the enlightened dismiss the referendum result as a statistical fluke, a terrible mistake which leaves the result open to challenge.
The research behind the pretty charts is not misreported. It’s just not being fully reported, especially if this dilutes the journalistic narrative. Let me illustrate using Lord Ashcroft’s “How did you vote” poll (fieldwork 21-23 June 2016). Unlike some other post-referendum polls, the full data tables have kindly been made available for scrutiny. Table A Part One shows the numbers answering “highest educational level achieved to date” question by whether they voted Leave or Remain. Part Two (row per cent) shows the proportion voting Leave or Remain within group; Part Three (column per cent) shows, for all Leave and Remain voters, the proportion falling within each of the groups.
Table A – Highest level of education that you have achieved to date
It is the second table that gets the headline: “Those with no formal education are twice as likely to vote leave at those with university degree/in education”. What proportion of Leave voters fall within this group? Just one per cent! This is an extreme example, but you get my point.
Let’s be absolutely clear: 99 per cent of Leave voters are misrepresented by this headline. Table A part 3 shows the main difference in Remain and Leave is between those educated to Secondary School standard (NVQ 1-3 equivalent) and those who obtained a University Degree (or equivalent NVQ). For older voters and voters in less wealthy areas, this is likely to be about access to higher education rather than educational potential, but again this doesn’t fit the prevailing narrative. Further work is needed to isolate this effect from other demographics such as age.
Similar analysis can be applied to age band and voting behaviour at the 2015 General Election. Taking the extremes of age, 73 per cent of 18-24 years voted Remain, while 60 per cent of those aged over 65 voted Leave. Hence the narrative that the pro-EU youth of our society have had their futures ruined by OAPs who don’t have to live with the consequences. What proportion of Leave and Remain voters are aged 18-24? Three per cent and eight per cent respectively. What proportion are aged over 65? Twenty-eight per cent and 20 per cent. Not such a good headline is it? In fact, according to this poll, the 65+ is the largest (modal) group in both the Leave and Remain vote. Another inconvenient perspective on the stats that gets in the way of a good story.
What about party affiliation? Ninety-six per cent of May 2015 UKIP voters voted for Leave. Hardly surprising – they did what it said on the tin. Yet UKIP voters make up only 25 per cent of the Leave vote. Most of the rest voted Conservative or Labour. Psephologists produce erudite-looking white papers containing scatter charts plotting UKIP voting by Local Authority in GE2015 against the Leave vote in June 2016. These compelling charts show about as strong a positive linear relationship you will get between a set of points in a statistical analysis. Unfortunately, though, this is the science of the bleedin’ obvious. UKIP was the only party providing a voice for anti-EU sentiment at GE2015. But just as the Tory vote does not equal the Tory membership, the Leave vote does not equal the UKIP vote.
Finally, to my second myth: that the vote was too close for the mandate to be meaningful. Table B shows the breakdown by Region.
Table B – Regional Breakdown
On a first-past-the-post basis this is a rout. Remain wins in only three of the 12 regions. This is an over-simplification, so I will refer to the excellent statistical modelling work by Chris Hanretty, Reader of Politics at the University of East Anglia who determined (based on modelling the actual local authority results down to constituency level) that if “Vote Leave” had been a political party it would probably have won 421 seats. A landslide representing 65 per cent of all seats (including Scotland) and 73 per cent of seats in England and Wales. If higher turnout in London and Scotland had tipped Remain over 50 per cent nationally, the result would have lacked a mandate in three-quarters of seats in England and Wales, leading to an historic democratic disaster.
One has to interrogate the data in different ways to see the balanced picture. Beware of self-appointed experts wielding statistics. The first question to ask is “what is the narrative they are peddling?” Statistics can be weaponised and poorly reported to support a weak argument. The pragmatic, maligned and much underestimated British electorate have known this for years. Is it such a surprise that so many have stopped listening?
The distressed heads of esteemed institutions in the statistics business such as the Royal Statistical Society who tweet about “post-truth politics” need to recognise that selective reporting of statistics is one of the main drivers of public cynicism towards experts. Caricatures make good headlines, but in the wrong hands they demonise voters. We need to call-out this sort of hyperbole for the nonsense it is.
Gary Bennett is CEO of The Stats People and is a statistical consultant to the survey research industry.
No wonder the British Public are suspicious of experts. Hardly a day goes by without another national media comment piece about how Brexit has divided the country, supported by colourful charts and scientific-looking analysis.
The Brexiteers, we are told, are lumpen, ageing, poorly educated, UKIP supporting, socially excluded provincials; the “left behind” as they have been tagged. The Remainers are young, optimistic, educated, supporters of progressive politics, living in London or metropolitan areas. The land’s ignorant failures and never-rans dragging the rest of the country screaming into the abyss is a compelling image.
The closeness of the Brexit vote share (Leavers falling just short of 52 per cent) is cited as evidence of a weak mandate for leaving the EU. Those supporting this narrative of the stupid versus the enlightened dismiss the referendum result as a statistical fluke, a terrible mistake which leaves the result open to challenge.
The research behind the pretty charts is not misreported. It’s just not being fully reported, especially if this dilutes the journalistic narrative. Let me illustrate using Lord Ashcroft’s “How did you vote” poll (fieldwork 21-23 June 2016). Unlike some other post-referendum polls, the full data tables have kindly been made available for scrutiny. Table A Part One shows the numbers answering “highest educational level achieved to date” question by whether they voted Leave or Remain. Part Two (row per cent) shows the proportion voting Leave or Remain within group; Part Three (column per cent) shows, for all Leave and Remain voters, the proportion falling within each of the groups.
Table A – Highest level of education that you have achieved to date
It is the second table that gets the headline: “Those with no formal education are twice as likely to vote leave at those with university degree/in education”. What proportion of Leave voters fall within this group? Just one per cent! This is an extreme example, but you get my point.
Let’s be absolutely clear: 99 per cent of Leave voters are misrepresented by this headline. Table A part 3 shows the main difference in Remain and Leave is between those educated to Secondary School standard (NVQ 1-3 equivalent) and those who obtained a University Degree (or equivalent NVQ). For older voters and voters in less wealthy areas, this is likely to be about access to higher education rather than educational potential, but again this doesn’t fit the prevailing narrative. Further work is needed to isolate this effect from other demographics such as age.
Similar analysis can be applied to age band and voting behaviour at the 2015 General Election. Taking the extremes of age, 73 per cent of 18-24 years voted Remain, while 60 per cent of those aged over 65 voted Leave. Hence the narrative that the pro-EU youth of our society have had their futures ruined by OAPs who don’t have to live with the consequences. What proportion of Leave and Remain voters are aged 18-24? Three per cent and eight per cent respectively. What proportion are aged over 65? Twenty-eight per cent and 20 per cent. Not such a good headline is it? In fact, according to this poll, the 65+ is the largest (modal) group in both the Leave and Remain vote. Another inconvenient perspective on the stats that gets in the way of a good story.
What about party affiliation? Ninety-six per cent of May 2015 UKIP voters voted for Leave. Hardly surprising – they did what it said on the tin. Yet UKIP voters make up only 25 per cent of the Leave vote. Most of the rest voted Conservative or Labour. Psephologists produce erudite-looking white papers containing scatter charts plotting UKIP voting by Local Authority in GE2015 against the Leave vote in June 2016. These compelling charts show about as strong a positive linear relationship you will get between a set of points in a statistical analysis. Unfortunately, though, this is the science of the bleedin’ obvious. UKIP was the only party providing a voice for anti-EU sentiment at GE2015. But just as the Tory vote does not equal the Tory membership, the Leave vote does not equal the UKIP vote.
Finally, to my second myth: that the vote was too close for the mandate to be meaningful. Table B shows the breakdown by Region.
Table B – Regional Breakdown
On a first-past-the-post basis this is a rout. Remain wins in only three of the 12 regions. This is an over-simplification, so I will refer to the excellent statistical modelling work by Chris Hanretty, Reader of Politics at the University of East Anglia who determined (based on modelling the actual local authority results down to constituency level) that if “Vote Leave” had been a political party it would probably have won 421 seats. A landslide representing 65 per cent of all seats (including Scotland) and 73 per cent of seats in England and Wales. If higher turnout in London and Scotland had tipped Remain over 50 per cent nationally, the result would have lacked a mandate in three-quarters of seats in England and Wales, leading to an historic democratic disaster.
One has to interrogate the data in different ways to see the balanced picture. Beware of self-appointed experts wielding statistics. The first question to ask is “what is the narrative they are peddling?” Statistics can be weaponised and poorly reported to support a weak argument. The pragmatic, maligned and much underestimated British electorate have known this for years. Is it such a surprise that so many have stopped listening?
The distressed heads of esteemed institutions in the statistics business such as the Royal Statistical Society who tweet about “post-truth politics” need to recognise that selective reporting of statistics is one of the main drivers of public cynicism towards experts. Caricatures make good headlines, but in the wrong hands they demonise voters. We need to call-out this sort of hyperbole for the nonsense it is.