Normal Deviate: Double Misunderstandings About p-values


sisyphean task

I’m really glad to see that the Normal Deviate has posted about the error in taking the p-value as any kind of conditional probability. I consider the “second” misunderstanding to be the (indirect) culprit behind the “first”.

Double Misunderstandings About p-values

March 14, 2013 – 7:57 pm

It’s been said a million times and in a million places that a p-value is not the probability of H0  given the data.

But there is a different type of confusion about p-values. This issue arose in a discussion on Andrew’s blog.

Andrew criticizes the New York times for giving a poor description of the meaning of p-values. Of course, I agree with him that being precise about these things is important. But, in reading the comments on Andrew’s blog, it occurred to me that there is often a double misunderstanding.

First, let me say that I am neither defending nor criticizing p-values in this post. I am just going to point out that there are really two misunderstandings floating around.

Two Misunderstandings

(1) The p-value is not the probability of H0 given the data.

(2) But neither is the p-value the probability of something conditional on H0 .

Deborah Mayo pointed this fact out in the discussion on Andrew’s blog (as did a few other people).

When we use p-values we are in frequentist-land. H0 (the null hypothesis) is not a random variable. It makes no sense to talk about the posterior probability of H0 . But it also makes no sense to talk about conditioning on H0 . You can only condition on things that were random in the first place.


You can read the rest of his post (with symbols intact) here.

Categories: P-values

Post navigation

3 thoughts on “Normal Deviate: Double Misunderstandings About p-values

  1. To avoid repetition, please comment over on “Normal Deviate’s” blog. Thanks.

  2. Corey

    That’s “Deviate”, not “Deviant”. 😉

    • Corey Bear: Corrected, but I seem to recall when he was discussing starting this blog with some of us at the Chancery Restaurant in London in June 2012, that he was toying with one of the two variants on names. So that’s my excuse for using either or both.

Blog at