Political Irony

Very interesting study done at Yale comes to some surprising conclusions about how our politics can override rational thinking.

The study did this by giving people a math problem to solve. The problem presented the results of a fictional medical trial for a new skin cream designed to treat rashes. Half the participants in the study were shown the following results:

On first glance it looks like more people who used the skin cream got better, but the real answer is based on whether your chance of getting better goes up if you use the skin cream. The correct solution is to compare those people who got better to those people who got worse.

People who used the skin cream had a 223:75 (less than 3 to 1) chance of getting better. But people who did not use the skin cream had a 107:21 (better than 5 to 1) chance of getting better. Put another way, 75% of the people who used the skin cream got better, but 84% (even more) of the people who did not use it got better. So the result is that you are more likely to get better if you do not use the skin cream. The confusion comes from the fact that more people (298) used the skin cream than did not (128), resulting in the misleading number that more people got better from the skin cream (if you ignore the number of people who got worse).

To remove any bias, they gave the other half of the participants the same problem with the results reversed, so the skin cream increased the chances of getting better.

This is actually a tricky problem to solve — 59% of the participants got the answer wrong.

They also grouped the participants in the study based on how numerate (“numerically literate” or “scientifically trained”) they are. Not surprisingly, numerate people did better on the problem.

Finally, they grouped the participants based on whether they are liberal democrats or conservative republicans. In this regard, there was no difference. Regardless of your politics, the more numerate you are the better you are at answering this question.

So far so good, but now we get to the interesting part of the study. They changed the problem, but only how they described it. Instead of a medical study, they presented it as the results of cities that had banned the concealed carrying of guns, and whether crime in these cities had gone up or down afterward.

Participants performed much differently on this version of the problem, even though all the numbers were the same. As in the previous version, they had two variants of the problem: one where the correct answer was that crime went down; and the other where crime went up.

Now here’s the ironic part. Here are two graphs plotting correct answers versus participant numeracy. The top one is for the skin cream / rash version, the bottom one is for the gun control / crime version:

Note how for the rash (top graph), the curves are pretty close, regardless of whether the participant is CR (conservative republican) or LD (liberal democrat), and also regardless of whether they solved the problem where the correct answer was that the cream decreased or increased the chances of the rash getting better.

But for crime (bottom graph) the participant’s answers strongly depended on their politics. Liberal democrats did best when the answer was that gun control made crime go down, while conservative republicans did best when the answer was that gun control made crime go up. So they did better if the answer agreed with their politics. But for people where the correct answer disagreed with their politics, the curves are mostly flat.

Ironically, the more numerate a participant is the more they are influenced by their politics. Numerate participants did 18% worse on the political problem. In fact the curves in the second graph are all over the place, sometimes even going down as numeracy goes up. It is difficult to guess what caused these crazy curves. If you are more scientifically literate, does it make you more sure of your answers, even when you are wrong? Are numerate people more likely to accept the incorrect answer on first glance, if it agrees with their politics? Or are they more willing to question an answer when it conflicts with their political beliefs?

At the very least it blows away the common idea that the smarter and more scientifically literate you are, the better you are at making correct decisions even when the right answer disagrees with your political beliefs. And this is true whether you are liberal or conservative.

This also helps explain why our country is so divided on the subject of climate change, regardless of the scientific research.

tl;dr — if this is too difficult to read and understand, here’s an infographic for you.