.
I gave a talk last week as part of the VT Department of Philosophy’s “brown bag” series. Here’s the blurb:
What is the Philosophy of Statistics? (and how I was drawn to it)
I give an introductory discussion of two key philosophical controversies in statistics in relation to today’s “replication crisis” in science: the role of probability, and the nature of evidence, in error-prone inference. I begin with a simple principle: We don’t have evidence for a claim C if little, if anything, has been done that would have found C false (or specifically flawed), even if it is. Along the way, I sprinkle in some autobiographical reflections.
My slides are at the end of this post:
First there was my accidentally taking a course in probability and statistics in the statistics department at Wharton as a graduate student. I was merely looking to keep up my math skills (from double majoring), and found the same book was being used in Statistics as in the Math Department.
Slides 14 and 15
I might never have sauntered into that first class on mathematical statistics had the Department of Statistics not been situated so closely to the Department of Philosophy [at U Penn]. …I suspected that understanding how these statistical methods worked would offer up solutions to the vexing problem of how we learn about the world in the face of error. (Mayo 1996, Error and the Growth of Experimental Knowledge: EGEK, Lakatos Prize 1998)
In writing Error and the Growth (EGEK), I was grappling with complaints from two sources:
Good
In statistics: the famous University Distinguished Professor I.J. Good demanding to know:
Why wasn’t I a Bayesian? (or a Doogian as he often said).
Laudan
In philosophy, leading Popperian Larry Laudan [1] wanted to know, given that confirmation theory was more or less discredited:
How does my work in philosophy of statistics make progress on their problems about large scale units: paradigms and research programs?
Slide 16: What I found then, still holds true:
Insights from grappling with foundational problems of statistics provide a gold mine for making progress on philosophical problems of evidence, inference, underdetermination, demarcation of science, and the general problems of the nature of error-prone inquiry. At the same time, statistics has long been the subject of philosophical debate marked by unusual heights of passion and controversy. There’s a “two-way street” between statistical science and philosophy of science.
In the discussion of my talk, I was asked for some autobiographical notes on I. J. Good, which I sidestepped to allow for other questions. Here are a few from the 80s, when I was barely out of graduate school. (1) Weirdly — don’t ask how — but me, I.J. Good, Persi Diaconis, and Patrick Suppes organized a session on Statistics and Parapsychology at the Popular Culture Association. Yes, really. Good was a believer. (2) Good’s license plate was 007. (3) Good occasionally published short notes from our informal discussions, often beginning “Lady Mayo asks…”. I guess I reminded him of someone. I learned most of what I call (statistical) “howlers” (criticisms of frequentist methods) from our lively debates, although in some cases it took a few years to solve them. (4) In 2006 when Sir David Cox attended the conference I ran with Aris Spanos, Error 06, I brought David over to meet I.J. Good (whose health was failing by then).
Good took credit or part credit for some of the major “paradoxes” or puzzles such as Jeffreys-(Good)-Lindley Paradox, and the stopping rule problem, and I’ve no reason to question this. On the latter (discussed in my last post), he wrote (in a paper from a conference we both attended):
In conversation I have emphasized to other statisticians, starting in 1950, that, in virtue of the ‘law of the iterated logarithm,’ by optional stopping an arbitrarily high sigma, and therefore an arbitrarily small tail-area probability, can be attained even when the null hypothesis is true. In other words if a Fisherian is prepared to use optional stopping (which usually he is not) he can be sure of rejecting a true null hypothesis provided that he is prepared to go on sampling for a long time. The way I usually express this ‘paradox’ is that a Fisherian [but not a Bayesian] can cheat by pretending he has a plane to catch like a gambler who leaves the table when he is ahead. (Good 1983, Good Thinking, p. 135)
One time, years later, I was driving him to his office, and said something like this:
You know Jack, as many times as I have heard you tell this, I’ve always been baffled as to its lesson about who is allowed to cheat. Error statisticians require the overall and not the ‘computed’ significance level be reported. To us, what would be cheating would be reporting the significance level you got after trying and trying again in just the same way as if the test had a fixed sample size. (A published version is on p. 351 of chapter 10 in EGEK: “Why you cannot be just a little bit Bayesian”*)
To my surprise, after pondering this a bit, Jack said something like, “Hmm, I never thought of it this way.” That, at any rate, is how I recount it in EGEK.
Anyway, I might hold some zoom forums in Phil Stat in late summer. If interested, write to jemille6@vt.edu with the subject PHILSTAT. If you want a recent syllabus, with all the needed links, there’s one from my 2023 seminar. My brown bag slides, but without the nifty animations are below.
[1] Many of us were working on developing a Center for Philosophy of Science at Virginia Tech akin to the one Laudan had left at Pittsburgh, but it wasn’t to be.
*Is it still true, if it ever was, that you cannot be just a little bit Bayesian? Share your thoughts and queries in the comments.
Error Statistics Philosophy 