bias table

He then asked people to select one of two possible conclusions relating to his fictitious study:

1: People who used the skin cream were more likely to get better than those who didn’t.

2: People who used the skin cream were more likely to get worse than those who didn’t.

The answer is not obvious, but if you look at the ratio of ‘rash got better’ ‘to rash got worse’, for each of the two groups, you realise that actually people who used the skin cream were more likely to get worse than those who didn’t. As a rule, those who were better at maths tended to get the answer right.

He then set precisely the same puzzle, but in a different context. Instead of rash and cream, the subject of the study concerned the link between carrying guns and crime in the US. In this case, reasoning went out of the window. If the numbers were fixed so that a quick look suggested carrying guns reduced crime – when a more careful look revealed the opposite finding – it was irrelevant how good the test subjects were at maths. If they held liberal view on crime, they were more likely to get the answer right. Then Kahan flipped the stats, to show the opposite finding. In this scenario, those with more conservative views on carrying guns got the answer right.

As it happens, psychologists have duplicated the experiment, but applied to the EU referendum. Their conclusion, quele surprise, “a voter’s ability to think rationally about the evidence for a referendum topic distinctly depends on whether or not that evidence supports their existing views,” – or so says The Online Privacy Foundation.

The research found that Leave voters tend to be more right wing, and more religious. Remain voters score higher in personality tests for openness. They are more concerned about privacy, and feel more strongly about human rights, or so claims

More controversially they say that Leave voters “typically have a worse grasp of statistical literacy, making them arguably less skilled at evaluating the pros and cons of important decisions such as financial investments.”

But is the Privacy Foundation being influenced by bias? They also say “Leave’ voters do significantly worse at interpreting the numbers than ‘Remain’ voters if those numbers show that immigration decreases crime. But they return to their normal performance if the numbers show that immigration increases crime.” But they add; “‘Remain’ voters typically have a much better grasp of how to accurately interpret statistics and numerical information. But, they also slip in performance if numerical evidence runs counter to their views. When the numbers show that immigration increases crime, they no longer beat the ‘Leavers’ in doing the maths.”

Let me finish by quoting from another experiment.

A paper by Linda Babcock and George Loewenstein examined ways to de-bias someone. They conducted an experiment, in which subjects played the role of either a plaintiff or defendant in a pretend trial involving a motor bike accident. The defendant was given a sum of money, and the plaintiff had to ask for damages to be paid from this money. In the event they were unable to agree, a third party, a judge, decided and the sum of money available was reduced. The experiment was re-run with different subjects.

Alas the two sides rarely agreed, each was convinced that the judge would decide in favour of their negotiating position. So Babcock and Loewenstein got subjects to read a paper about self-serving bias, and even tested them on it to ensure they understood. The result: after reading the paper, subjects were even less likely to agree as they became convinced the other party was biased.

It turned out the only solution was to get each party to argue the other person’s case. Only then was it more likely that the two parties would agree on a settlement without calling on the services of a judge, who reduced the overall amount of money available.

Curiously, or so it is said, Boris Johnson did something similar before deciding on which side of the referendum he stood. He wrote two articles, one for, one against. He then selected the Leave article for his weekly Telegraph column, rejecting the Remain piece, presumably to the bin. According to Andrew Marr, speaking on his Sunday morning TV show, rumour has it, that the Remain article made the more convincing case.

Or, it this all too biased to say?

By Michael Baxter, economics blogger