I’m really glad to see that the Normal Deviate has posted about the error in taking the p-value as any kind of conditional probability. I consider the “second” misunderstanding to be the (indirect) culprit behind the “first”.
Double Misunderstandings About p-values
March 14, 2013 – 7:57 pm
It’s been said a million times and in a million places that a p-value is not the probability of H0 given the data.
But there is a different type of confusion about p-values. This issue arose in a discussion on Andrew’s blog.
Andrew criticizes the New York times for giving a poor description of the meaning of p-values. Of course, I agree with him that being precise about these things is important. But, in reading the comments on Andrew’s blog, it occurred to me that there is often a double misunderstanding.
First, let me say that I am neither defending nor criticizing p-values in this post. I am just going to point out that there are really two misunderstandings floating around.
(1) The p-value is not the probability of H0 given the data.
(2) But neither is the p-value the probability of something conditional on H0 .
Deborah Mayo pointed this fact out in the discussion on Andrew’s blog (as did a few other people).
When we use p-values we are in frequentist-land. H0 (the null hypothesis) is not a random variable. It makes no sense to talk about the posterior probability of H0 . But it also makes no sense to talk about conditioning on H0 . You can only condition on things that were random in the first place.
You can read the rest of his post (with symbols intact) here.
To avoid repetition, please comment over on “Normal Deviate’s” blog. Thanks.
That’s “Deviate”, not “Deviant”. 😉