The fourth meeting of our New Phil Stat Forum*:
The Statistics Wars
and Their Casualties
January 7, 16:00 – 17:30 (London time)
11 am-12:30 pm (New York, ET)**
**note time modification and date change
Putting the Brakes on the Breakthrough,
or “How I used simple logic to uncover a flaw in a controversial 60-year old ‘theorem’ in statistical foundations”
Deborah G. Mayo
.
HOW TO JOIN US: SEE THIS LINK
ABSTRACT: An essential component of inference based on familiar frequentist (error statistical) notions p-values, statistical significance and confidence levels, is the relevant sampling distribution (hence the term sampling theory). This results in violations of a principle known as the strong likelihood principle (SLP), or just the likelihood principle (LP), which says, in effect, that outcomes other than those observed are irrelevant for inferences within a statistical model. Now Allan Birnbaum was a frequentist (error statistician), but he found himself in a predicament: He seemed to have shown that the LP follows from uncontroversial frequentist principles! Bayesians, such as Savage, heralded his result as a “breakthrough in statistics”! But there’s a flaw in the “proof”, and that’s what I aim to show in my presentation by means of 3 simple examples:
- Example 1: Trying and Trying Again
- Example 2: Two instruments with different precisions
(you shouldn’t get credit/blame for something you didn’t do) - The Breakthrough: Don’t Birnbaumize that data my friend
As in the last 9 years, I posted an imaginary dialogue (here) with Allan Birnbaum at the stroke of midnight, New Year’s Eve, and this will be relevant for the talk.
The Phil Stat Forum schedule is at the Phil-Stat-Wars.com blog
Readings:
One of the following 3 papers:
My earliest treatment via counterexample:
- Mayo, D. G. (2010). “An Error in the Argument from Conditionality and Sufficiency to the Likelihood Principle” in Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability and the Objectivity and Rationality of Science (D Mayo and A. Spanos eds.), Cambridge: Cambridge University Press: 305-14.
A deeper argument can be found in:
- Mayo 2014. “On the Birnbaum Argument for the Strong Likelihood Principle,” (with discussion & rejoinder)Statistical Science, 29(2), 227-239, 261-266.
For an intermediate Goldilocks version (based on a presentation given at the JSM 2013):
- Mayo 2013. “Presented Version: On the Birnbaum Argument for the Strong Likelihood Principle.” In JSM Proceedings, Section on Bayesian Statistical Science. Alexandria, VA: American Statistical Association, 440-453.
This post from the Error Statistics Philosophy blog will get you oriented. (It has links to other posts on the LP & Birnbaum, as well as background readings/discussions for those who want to dive deeper into the topic.)
Slides and Video Links:
D. Mayo’s slides: “Putting the Brakes on the Breakthrough, or ‘How I used simple logic to uncover a flaw in a controversial 60-year old ‘theorem’ in statistical foundations’”
D. Mayo’s presentation:
- (Link to paste in browser): https://philstatwars.files.wordpress.com/2021/01/mayo_172021_presentation.mp4
- SHORT LINK (quick): https://wp.me/abBgTB-x4
Discussion on Mayo’s presentation:
- (Link to paste in browser): https://philstatwars.files.wordpress.com/2021/01/mayo-172021-discussion-1.mp4
- SHORT LINK (quick): https://wp.me/abBgTB-xc
Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting.
You may wish to look at my rejoinder to a number of statisticians: Rejoinder “On the Birnbaum Argument for the Strong Likelihood Principle”. (It is also above in the link to the complete discussion in the 3rd reading option.)
I often find it useful to look at other treatments. So I put together this short supplement to glance through to clarify a few select points.
Please post comments on the Phil Stat Wars blog here.
Error Statistics Philosophy 
Thank you very much for your wonderful papers and presentation.
Yesterday, I commented in the seminar that the two likelihoods in your Example1 aren’t proportional.
But as you replied, the example(ii) in Cox(1978; p.53) is the same as your Example1, and it’s said that the two likelihoods are identical.
A Japanese mathematician, Prof. Gen Kuroki, also points out to me in Twitter that the two likelihoods in your Example1 are identical.
I am now investigating whether I told a lie or not, but I am very confused. If someone know more information or mathematical derivations, I would like them to show the hint to me.
Sorry for this confusion.
Now, I understand I was wrong.
A Japanese mathematician, Prof. Gen Kuroki, explains to me why the two likelihoods in your Example1 become identical.
In Q&A time at yesterday’s session, I told false information (I told that two likelihoods aren’t proportional).
I am very sorry for my misunderstanding.
If you are interested in the reason why I misunderstood, I would like to explain it somewhere.
Yusuke: A lot of people get this wrong, which is why I showed the equation from Cox and Hinkley (1974) in the “supplement” I prepared for my presentation yesterday. The thing is that the example is given by BOTH sides, so if there were anything wrong with it it would be a problem for them in their choice of example. There are zillions of LP violations: any use of a confidence level, p-value, standard error, error probability will do, and one doesn’t even need to name an example to make the points on either side. I used this deramatic example because, amazingly enough, it’s one the pro-LP people are happy about (i.e., they don’t think it should matter if you’re guaranteed to reject a null hypothesis erroneously). Less dramatic examples about (e.g., binomial vs negative binomial). It seemed quicker in a presentation to have an example where the LP violation was a difference in p-values, rather than keep saying “an LP violation”.
If you want to explain your point, you are welcome to do so in a comment here. Thank you for your interest.
Thank you for your reply, and I am sorry again for my confusion.
Let me just explain where I was wrong. I calculated the density function conditioned on n (, Pr(X1 = x1, X2 = x2, …, Xn = xn | N = n),) also for the Trying and Trying Again experiments. I should have not conditioned on n. The density function which I need to calculate may be denoted by Pr(X1 = x1, X2 = x2, …, Xn = xn, N = n). This unconditional likelihood becomes identical to the one for the fixed-n experiment.
This discussion and any additional questions are and will be at phil-stat-wars.com. I just replied to a comment that had been in spam:
https://phil-stat-wars.com/2020/12/03/january-7-on-the-birnbaum-argument-for-the-strong-likelihood-principle-deborah-mayo/comment-page-1/#comment-166