Do political beliefs affect our ability to crunch numbers?
Our brain is not straightforward. The past few years have served as a wake-up call for people who had not realized that believing weird things is quite common. From the rise of QAnon to the politicized debates over mask-wearing and the safety of the COVID-19 vaccines, it has become abundantly transparent that our thinking easily zigs and zags according to our preconceptions. But what about math?
I am fascinated by the work that Dan Kahan and his colleagues publish. They often look at the interplay between beliefs and group membership. We are very tribal by nature, and if our tribe tells us to believe that the latest election was stolen by the other tribe, we can adopt this thinking without even looking at the evidence. After all, what is worse: believing something that isn’t true or believing something that goes against what your community cherishes and risk getting kicked out?
In 2013, Kahan and collaborators devised to detect the impact of Americans’ political outlook on number crunching. It involved a new skin cream. Participants were told that this new cream was supposed to treat skin rashes. Of course, the big question was: does the cream work? The text participants had to read revealed that the cream had been tested in a trial. Some of the people used the cream for two weeks; others did not. Some rashes got better; others did not.
Participants were presented with a simple table that showed how many people had rashes that improved versus did not improve among those who used the cream and those that did not.
Table adapted from Figure 2 from Kahan et al. (2013) Behaviour Public Policy
By looking at these numbers, can you conclude if the cream worked or not? It turns out that this little exercise is not easy. Most people make a mistake when interpreting these numbers, as they did in the experiment itself. If you only look at the top row, you see more people who got better than people who got worse when the cream was used, and you conclude that the cream worked. But you’re not taking into account the control group of people who did not use the cream.
If you simply compare the 223 people who got better with the cream to the 107 who got better without the cream, you conclude that the cream worked. But you’re not taking into account the total number of people in each group: 298 tried the cream, while only 128 did not.
The correct way to interpret the table is to calculate the percentage of people who got better in both groups and to compare these percentages. Among those who used the cream, 223 divided by 298 (0.75 or 75%) got better. Among those who did not use the cream, 107 divided by 128 (0.84 or 84%) got better. In this simplistic example that is stripped of many statistical nuances, the cream clearly did not work.
The ability to use this table to arrive at the correct answer is part of a set of skills called numeracy. In much the same way that literacy means an ability to read and write based on an understanding of the alphabet, numeracy is our ability to interpret and work with numbers, and the numbers on numeracy are not encouraging. According to and released in 2013, Canada ranks similarly in terms of numeracy to the United States and the United Kingdom. Only 13% of Canadians (or about 1 in 8) can understand complex mathematical information and work with mathematical arguments. Nearly 1 in 4 Canadians can either only perform simple or very simple one-step mathematical operations, like counting or ordering. This is something worth keeping in mind when armchair experts on social media have “done their own research” on COVID-19 numbers and have come to an alarming conclusion that goes against the consensus.
But the skin cream experiment is a lot more than yet another depressing example of our numeracy challenges. Some of the participants got the skin cream story. Others got a different story. It was about a law banning private citizens from carrying concealed handguns in public. Some cities had this ban in place, while others did not. Some cities saw a decrease in crime, while others did not. You probably know where this is going.
The same table as above was presented to them. The numbers were identical but the labels changed. Some cities with the ban saw an increase in crime while others saw a decrease, and some cities that did not enact the ban likewise saw an increase or a decrease in criminality. Did the ban work?
According to the numbers, the ban did not work. Crime increased more in the cities that had adopted this handgun ban. But political affiliation played a trick on the study participants. Before they were presented with the fictional scenario, participants were asked to fill out standard questionnaires to determine their political affiliations and their numeracy. When the researchers looked at their conservative Republican participants, the higher their numeracy score (i.e. the better they were at understanding and playing with numbers in general), the more likely they were to report that the ban had indeed not worked. The most numerate among them would crunch the numbers the right way, calculate percentages, and see the true answer.
But for the liberal Democrat participants? Their numeracy score did not help them. Whether they were good with numbers or not had little impact on their conclusion. They looked at that table and fewer than half reported that the ban had not worked.
Before conservative readers use this as evidence of superiority, the researchers of course flipped the script. Some of the research participants saw the exact same numbers but labelled in such a way that made the ban work in reducing crime. The results were flipped as well. Democrats were better at figuring it out if they were good at math, but math skills did not help Republicans much. If instead of a handgun ban they had been shown data about skin cream, there was no real difference between Democrats and Republicans. Both groups were better at figuring out the real answer if their mathematical ability was high, as long as the topic was a non-political topical ointment.
This study provides one more piece of evidence in a larger body of work that shows us how important protecting our identity is. If I belong to an important group that believes that banning handguns is wrong and would not work to reduce crime, and I am faced with data that points in the other direction, I may be inclined to not think about the data too much.
As we saw earlier with the skin cream example, it is so tempting to glance at the numbers and draw a quick (but wrong) conclusion. We need to shift to a more analytical gear in our brain to fully consider what to do with the numbers to arrive at the right conclusion. If my tribal affiliation is more important to me than arriving at the truth, I can take a shortcut and report that the data agrees with me. But if it’s the shortcut that threatens my group’s beliefs and I am good enough at math, I can pause, rethink the problem, and calculate the percentages to arrive at an answer that is both accurate and acceptable to my tribe.
We are all susceptible to this “motivated numeracy,” as it has been called. It is sobering and nerve-racking to realize that numbers can so easily be analyzed by the brain in whichever way reassures us.
Take-home message:
-Numeracy is the ability to engage with numbers and information that contains numbers in a range of situations in everyday life.
-Motivated numeracy is when the brain starts with a conclusion it likes and treats numbers in a way that will conform to this conclusion.
