I’m thinking of buying this book.
On the face of it it seems like the kind of book I’ll enjoy. Admittedly I’d enjoy a nice biography about Metallica or some other heavy rock band more, but I need to maintain the façade of loving statistics. It seems to be an historical account of why we use significance testing and why it’s a terrible idea. I’m fairly confident that I already know most of what it will say, but the synopsis promises some nice discipline-specific examples of the ‘train wreck’ that is hypothesis testing. I probably won’t know these and it seems like there’s potential for entertainment. However, the two reviews of this book (which are fairly positive) say the following:
Reviewer 1: “In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.”
Reviewer 2: “A relationship between two variables is statistically significant if there is a low probability (usually less than five per cent) of it happening by chance.”
Both of which are wrong. A result is statistically significant if the observed effect/relationship/whatever is unlikely to have occurred GIVEN THAT THERE IS NO EFFECT/RELATIONSHIP/WHATEVER IN THE POPULATION.
So, although I probably will buy this book because it looks interesting, I offer up my own free version here in this blog. I know it’s insanely generous of me to give you a whole book for free, but I’m a caring kind of guy. So here it is:
The Cult of Statistical Significance and Why it Will Fry Your Brain
By
Andy P. Field
If people can read an entire book about a concept and still not understand what it is, then that concept is probably unnecessarily confusing, poorly conceived and should be buried in a lead box, chained with particularly curmudgeonly rattlesnakes, guarded by rabid hounds, and placed in Satan’s toilet bowl. Only someone with the guile of Indiana Jones should be able to retrieve it, and should they ever manage to, this person should be dipped in sugar and set upon by locusts. It turns out that significance testing is such a concept (although with N = 2 I probably didn't have enough power to significance test my hypothesis).
The End
