Next Phil Stat Forum: January 7: D. Mayo: Putting the Brakes on the Breakthrough (or “How I used simple logic to uncover a flaw in …..statistical foundations”)

The fourth meeting of our New Phil Stat Forum*:

The Statistics Wars
and Their Casualties

January 7, 16:00 – 17:30  (London time)
11 am-12:30 pm (New York, ET)**
**note time modification and date change

Putting the Brakes on the Breakthrough,

or “How I used simple logic to uncover a flaw in a controversial 60-year old ‘theorem’ in statistical foundations” 

Deborah G. Mayo



ABSTRACT: An essential component of inference based on familiar frequentist (error statistical) notions p-values, statistical significance and confidence levels, is the relevant sampling distribution (hence the term sampling theory). This results in violations of a principle known as the strong likelihood principle (SLP), or just the likelihood principle (LP), which says, in effect, that outcomes other than those observed are irrelevant for inferences within a statistical model. Now Allan Birnbaum was a frequentist (error statistician), but he found himself in a predicament: He seemed to have shown that the LP follows from uncontroversial frequentist principles! Bayesians, such as Savage, heralded his result as a “breakthrough in statistics”! But there’s a flaw in the “proof”, and that’s what I aim to show in my presentation by means of 3 simple examples:

  • Example 1: Trying and Trying Again
  • Example 2: Two instruments with different precisions
    (you shouldn’t get credit/blame for something you didn’t do)
  • The Breakthrough: Don’t Birnbaumize that data my friend

As in the last 9 years, I posted an imaginary dialogue (here) with Allan Birnbaum at the stroke of midnight, New Year’s Eve, and this will be relevant for the talk.

The Phil Stat Forum schedule is at the blog 


One of the following 3 papers:

My earliest treatment via counterexample:

A deeper argument can be found in:

For an intermediate Goldilocks version (based on a presentation given at the JSM 2013):

This post from the Error Statistics Philosophy blog will get you oriented. (It has links to other posts on the LP & Birnbaum, as well as background readings/discussions for those who want to dive deeper into the topic.)

Slides and Video Links:

D. Mayo’s slides: “Putting the Brakes on the Breakthrough, or ‘How I used simple logic to uncover a flaw in a controversial 60-year old ‘theorem’ in statistical foundations’”

D. Mayo’s  presentation:

Discussion on Mayo’s presentation:

Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting.

You may wish to look at my rejoinder to a number of statisticians: Rejoinder “On the Birnbaum Argument for the Strong Likelihood Principle”. (It is also above in the link to the complete discussion in the 3rd reading option.)

I often find it useful to look at other treatments. So I put together this short supplement to glance through to clarify a few select points.

Please post comments on the Phil Stat Wars blog here.


Categories: Birnbaum, Birnbaum Brakes, Likelihood Principle | 5 Comments

Post navigation

5 thoughts on “Next Phil Stat Forum: January 7: D. Mayo: Putting the Brakes on the Breakthrough (or “How I used simple logic to uncover a flaw in …..statistical foundations”)

  1. Yusuke Ono

    Thank you very much for your wonderful papers and presentation.

    Yesterday, I commented in the seminar that the two likelihoods in your Example1 aren’t proportional.

    But as you replied, the example(ii) in Cox(1978; p.53) is the same as your Example1, and it’s said that the two likelihoods are identical.

    A Japanese mathematician, Prof. Gen Kuroki, also points out to me in Twitter that the two likelihoods in your Example1 are identical.

    I am now investigating whether I told a lie or not, but I am very confused. If someone know more information or mathematical derivations, I would like them to show the hint to me.

    Sorry for this confusion.

    • Yusuke Ono

      Now, I understand I was wrong.

      A Japanese mathematician, Prof. Gen Kuroki, explains to me why the two likelihoods in your Example1 become identical.

      In Q&A time at yesterday’s session, I told false information (I told that two likelihoods aren’t proportional).

      I am very sorry for my misunderstanding.

      If you are interested in the reason why I misunderstood, I would like to explain it somewhere.

      • Yusuke: A lot of people get this wrong, which is why I showed the equation from Cox and Hinkley (1974) in the “supplement” I prepared for my presentation yesterday. The thing is that the example is given by BOTH sides, so if there were anything wrong with it it would be a problem for them in their choice of example. There are zillions of LP violations: any use of a confidence level, p-value, standard error, error probability will do, and one doesn’t even need to name an example to make the points on either side. I used this deramatic example because, amazingly enough, it’s one the pro-LP people are happy about (i.e., they don’t think it should matter if you’re guaranteed to reject a null hypothesis erroneously). Less dramatic examples about (e.g., binomial vs negative binomial). It seemed quicker in a presentation to have an example where the LP violation was a difference in p-values, rather than keep saying “an LP violation”.
        If you want to explain your point, you are welcome to do so in a comment here. Thank you for your interest.

        • Yusuke Ono

          Thank you for your reply, and I am sorry again for my confusion.

          Let me just explain where I was wrong. I calculated the density function conditioned on n (, Pr(X1 = x1, X2 = x2, …, Xn = xn | N = n),) also for the Trying and Trying Again experiments. I should have not conditioned on n. The density function which I need to calculate may be denoted by Pr(X1 = x1, X2 = x2, …, Xn = xn, N = n). This unconditional likelihood becomes identical to the one for the fixed-n experiment.

  2. This discussion and any additional questions are and will be at I just replied to a comment that had been in spam:

I welcome constructive comments that are of relevance to the post and the discussion, and discourage detours into irrelevant topics, however interesting, or unconstructive declarations that "you (or they) are just all wrong". If you want to correct or remove a comment, send me an e-mail. If readers have already replied to the comment, you may be asked to replace it to retain comprehension.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at