In the research, by Yale law professor Dan Kahan, he gave identical numerical data supposedly coming from an actual survey to a sample of over 1000 people. In one of the sets of data it was described as testing skin cream effectiveness. In the other, it was testing the effectiveness of bans on concealed gun permits.
In each case the exact same numbers were given in two different ways, one that showed improved results, and one that showed lack of improvement.
In the skin cream problems, the results showed that "more numerate" people were much more likely to get the right answer. There also was no significant difference between liberals and conservatives of similar numeracy.
But here is how Chris Mooney described the results in a Mother Jones article:
"So how did people fare on the handgun version of the problem? They performed quite differently than on the skin cream version, and strong political patterns emerged in the results—especially among people who are good at mathematical reasoning. Most strikingly, highly numerate liberal Democrats did almost perfectly when the right answer was that the concealed weapons ban does indeed work to decrease crime (version C of the experiment)—an outcome that favors their pro-gun-control predilections. But they did much worse when the correct answer was that crime increases in cities that enact the ban (version D of the experiment).
The opposite was true for highly numerate conservative Republicans: They did just great when the right answer was that the ban didn't work (version D), but poorly when the right answer was that it did (version C).
A graph of the results given in the article may raise questions about the "numeracy" classifications, or perhaps liberals only develop bias above numeracy level 5.5 while conservatives bias runs across a wider range of ability. But they clearly support the idea that they gave the answer they wanted, rather than the one they surely knew to be true.
Moody concluded with a telling statement, "The Scottish Enlightenment philosopher David Hume famously described reason as a "slave of the passions." Today's political scientists and political psychologists, like Kahan, are now affirming Hume's statement with reams of new data. This new study is just one out of many in this respect, but it provides perhaps the most striking demonstration yet of just how motivated, just how biased, reasoning can be—especially about politics."
"he gave identical numerical data supposedly coming from an actual survey"
ReplyDeleteIn other words, he lied to the subjects. Did it say in their consent forms that they might be lied to? Maybe some of them (on both sides) were using reason, and deciding that the data was wrong. (Like we do when a die comes up heads too many times.)
Sue,
ReplyDeleteI think the data was presented as a problem to be analyzed in conjunction with the numeracy testing.
I don't know if they told them it was actual research data or simulated, but either way, I'm not sure that "deciding the data was wrong" doesn't seem like political bias to me. They seemed to have no such questions about face-cream. The pattern of wrong data followed very political, not mathematical patterns.
The face cream was mythical. The issues addressed by the other data were clearly real.
ReplyDeleteI'm not saying this issue (of our politics interfering with our judgment) doesn't exist. I'm just not impressed with this test of it. If I were one of the subjects, I might write a note saying I thought there was a problem with the data, and I wanted to see the original study design.
Hmm, now I'm wondering if there's any way to test this properly.
ReplyDelete