PrawfsBlawg: A Misguided Philosophy of Science

Susan Haack provides a nice way to think about how we should approach knowledge production in the social sciences. She equates knowledge to a crossword puzzle. A single word, all alone in the grid, is not well-supported (especially if, as most scientific questions likely are, it is a Saturday NY Times). If 35-down is a seven-letter word for "Oceanographers' references," SEAMAPS fits, but we might not be all that confident it is right. But then we look at 46-across, which starts on the M, and we think that MATED could be the right answer for "Like shoes and socks;" we become a bit more sure that SEAMAPS is right. And our faith in MATED grows when we realize that 47-down, which starts on the D, is probably "DOSE," since the clue is "Recommended intake." So DOSE provides warrant for MATED, which in turn warrants SEAMAPS. Once the whole puzzle is filled, we're pretty sure we're right. We could be completely wrong--I've certainly erased entire quadrants of a crossword puzzle before--but the odds are low.

Highlighted by cgibbons

I attended a conference several years ago in which one of the speakers said that if we added up every paper that said "factor x is responsible for y% of the crime drop in the 1990s," we would see that we've explained about 250% of the crime decline. Everyone laughed but thought little of it. But this is a huge problem: the words in the crossword puzzle are not crossing properly. Something, somewhere, is wrong, but by taking a corpuscular view of empirical evidence, we miss it.

Highlighted by cgibbons

but by thinking in terms of refutation, we provide the wrong results in our papers. In an inductionist world, p-values and t-statistics mean very little. What matters are confidence intervals, and these are almost never repoted.

Highlighted by cgibbons