Normal Deviate: Double Misunderstandings About p-values


sisyphean task

I’m really glad to see that the Normal Deviate has posted about the error in taking the p-value as any kind of conditional probability. I consider the “second” misunderstanding to be the (indirect) culprit behind the “first”.

Double Misunderstandings About p-values

March 14, 2013 – 7:57 pm

It’s been said a million times and in a million places that a p-value is not the probability of H0  given the data.

But there is a different type of confusion about p-values. This issue arose in a discussion on Andrew’s blog.

Andrew criticizes the New York times for giving a poor description of the meaning of p-values. Of course, I agree with him that being precise about these things is important. But, in reading the comments on Andrew’s blog, it occurred to me that there is often a double misunderstanding.

First, let me say that I am neither defending nor criticizing p-values in this post. I am just going to point out that there are really two misunderstandings floating around.

Two Misunderstandings

(1) The p-value is not the probability of H0 given the data.

(2) But neither is the p-value the probability of something conditional on H0 .

Deborah Mayo pointed this fact out in the discussion on Andrew’s blog (as did a few other people).

When we use p-values we are in frequentist-land. H0 (the null hypothesis) is not a random variable. It makes no sense to talk about the posterior probability of H0 . But it also makes no sense to talk about conditioning on H0 . You can only condition on things that were random in the first place.


You can read the rest of his post (with symbols intact) here.

Categories: P-values | 3 Comments

Post navigation

3 thoughts on “Normal Deviate: Double Misunderstandings About p-values

  1. To avoid repetition, please comment over on “Normal Deviate’s” blog. Thanks.

  2. Corey

    That’s “Deviate”, not “Deviant”. 😉

    • Corey Bear: Corrected, but I seem to recall when he was discussing starting this blog with some of us at the Chancery Restaurant in London in June 2012, that he was toying with one of the two variants on names. So that’s my excuse for using either or both.

I welcome constructive comments that are of relevance to the post and the discussion, and discourage detours into irrelevant topics, however interesting, or unconstructive declarations that "you (or they) are just all wrong". If you want to correct or remove a comment, send me an e-mail. If readers have already replied to the comment, you may be asked to replace it to retain comprehension.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

%d bloggers like this: