While I would agree that there are differences between Bayesian statisticians and Bayesian philosophers, those differences don’t line up with the ones drawn by Jon Williamson in his presentation to our Phil Stat Wars Forum (May 20 slides). I hope Bayesians (statisticians, or more generally, practitioners, and philosophers) will weigh in on this.
1 Are the wars mostly in statistics, not philosophy? According to Williamson, Bayesian philosophers, while mostly subjective, just “see Bayesianism as being concerned with belief, and so not a rival” to frequentist statistics, while Bayesian statisticians do “see Bayesianism as a rival to frequentist statistical inference—‘statistics wars’.” [SLIDE 6]. So, in his view, philosophers of statistics don’t see a war between frequentists and Bayesians whereas statisticians do. I think, if anything, it is the reverse. Bayesian statisticians are more eclectic, and much less inclined to see a war between frequentists and Bayesians. When they do so, they are, largely, wearing philosophical hats. Granted they often do not recognize that they’re making philosophical presuppositions when waging a war with frequentists, (error statisticians in my terminology), but by and large they are happy to get on with the job. By contrast, philosophers who accept a frequentist, error statistical view are generally seen as exiles, under the presumption that only Bayesianism gives a sound (coherent) underlying philosophy.[i] (That is why the alt name for my blog is “frequentists in exile”.)
I’m not saying, by the way, that the main stat wars are between frequentists and Bayesians–there are many other battles as well. I’m just addressing some of the points Williamson makes.
I also find it surprising that according to Williamson, a Bayesian statistician appears to be much more of a true blue subjective (in the manner of de Finetti) than is the Bayesian philosopher. Again, I would have thought it was the opposite. [SLIDE 6] Statisticians seem much more eclectic than philosophers in their interpretations of probability (see Gelman and Shalizi 2013). Williamson avers that Bayesian statisticians “often doubt the existence of non-epistemic probabilities (following Bruno de Finetti)”. Non-epistemic probabilities” he says are “generic frequencies or single-case chances”. Doubting the existence of frequencies and chances, the Bayesian statistician, he claims, does not seek to “directly calibrate one’s credences [degrees of belief] to nonepistemic probabilities (generic frequencies or single-case chances).”
But certainly the large class of non-subjective, default, reference, and empirical Bayesians go beyond probability as subjective degrees of belief, and increasingly separate themselves from traditional subjective and personalist philosophies. Being calibrated to frequencies in some sense, I thought, was one of the main advantages of non-subjective, objective, or default Bayesianism in statistical practice. This is so, regardless of their metaphysics on chances or propensities: it suffices to allude to relative frequencies (actual or hypothetical) stemming from modeled phenomena or deliberately designed experiments.
Having a battle about where the wars are–statistics or philosophy of statistics–may not be productive, but I think it’s very important to understand the nature of the debates. In fact, one of the most serious casualties of the statistics wars from the philosophical perspective is in obscuring the roles of statistical methods (and other formal and quasi-formal methodologies) in addressing the epistemological problem of how to generate, learn and generalize from data. In other words, the wars have confused the value of statistics for philosophy.
2. Philosophy of confirmation vs philosophy of statistics. Now there is a radical difference to which Williamson’s discussion points, and it is between a given project–it might be called Bayesian confirmation theory, Bayesian epistemology, inductive logic or the like–and what I would call philosophy of statistics, or the philosophy of inductive-statistical inference. Bayesian confirmation theorists, since Carnap, have a tradition of building an account based on a restricted language: statements, propositions, and first order logics. By contrast, statisticians and statistical philosophers refer to probability models, continuous random variables, parameters and the like. The philosophical project is essentially to justify a mode of inductive inference, basically, Carnap’s straight rule or a version of enumerative induction: if k% of A’s have been B’s then believe the next A will be B to degree k. Perhaps a stipulation that the observations satisfy a condition of randomness or exchangeability is added.
An example from Williamson (SLIDE 7) is this: Suppose your evidence E is: 17 of a random sample of a hundred 21-year-olds develop a cough. That the sample frequency is 0.17 is evidence that the chance/frequency is ≈ 0.17. A case of what he calls a direct inference would be to take .17 as how confident one should be, or how strongly one should believe, the statement A: that Cheesewright, who is 21, gets a cough. In the confirmation philosopher’s project, there is a restriction to a finite language, set of predicates, and assignments of probabilities to chosen elements. A statistician, instead, might appeal to Bernouilli trials and a Binomial model of experiment to reach such a (direct) inference to the probability of an event occurring on the next trial.
Philosophers of confirmation sought an a priori (non empirical) way to justify such an inference–to solve the traditional problem of induction–and any reference to a probability model, or even slipping in that it’s a random sample, makes an empirical assumption. So they shied away from tackling the inductive problem using models. By the way, I don’t view inductive inference as inferring probabilities of claims, but making inferences that go beyond the data–they are ampliative. They are qualified using probabilities, but these are not posteriors in hypotheses. Even falsification requires inductive inferences in this sense (see SIST, excursion 2 Tour II, p. 83).
I don’t think philosophers still consider that an a priori justification of inductive inference is possible or even desirable. Thus, the impetus to restricting the philosophical account of inference to specially crafted first order languages goes by the board, and we can freely talk about design-based or model based probabilities. Unlike what the confirmation project typically supposes, appeals to such models may be warranted or, alternatively, falsified. Showing how is part of what is involved in solving the problem of induction now, or so I argue (2018, pp 107-115).
3. Do statisticians not move from general probabilities to specific assignments? It’s interesting that Williamson claims that “statisticians tend not to appeal to direct inference principles” that move from population probabilities and frequencies to degrees of probability, belief, or support in a particular case. The reason, if I understand him, is that they tend not to believe in these non-epistemic, frequentist probability notions–taking us back to point 1 above.
I find Bayesian statisticians/practitioners highly interested in assigning degrees of belief, credence, or plausibility to events– whether we consider that tantamount to assigning beliefs to statements about events, or to events defined in a model. In fact Bayesian statisticians see a key selling point of their methodology that it offers a way to assign probabilities to particular events and hypotheses, whereas the frequentist error statistician generally only speaks of the performance properties of methods in repetitions. The error statistician is also largely interested in inverse inference from data to claims about aspects of the data generating method (rather than direct inference).
The latest move (by both Bayesian and frequentist practitioners) to embrace what I call a “screening model” of statistical significance tests would seen to be an example of practitioners performing direct inferences. (See SIST, excursion 5 Tour II.) Here the probability of a particular hypothesis is given by considering it to have been (randomly?) selected from a universe or urn of hypotheses (where it’s assumed some % are known to be true).
Williamson himself appeals to confidence intervals in illustrating a direct inference from a confidence level to a degree of belief in a particular interval estimate. A popular reconciliation, which I think he endorses, makes use of frequentist matching priors. Fisher’s fiducialist essentially tried to do this without appealing to priors. So again it’s not clear why he takes statisticians as uninterested in direct inference. True, contradictions result from probabilistic instantiation, as noted in my “casualties“, and in Williamson’s presentation. (His solution is to drop Bayesian conditioning and start over with new maximum entropy priors.) The error statistician will also take error probabilities of methods as qualifying a particular inference. But it is qualifying, not its degree of believability, but rather, how well tested or corroborated it is.
The upshot: I consider statistical methods far more relevant to the philosophers’ epistemological projects than I find in Williamson’s portrayal, at least based on his May 20 presentation. Philosophers of confirmation and formal epistemology shortchange themselves by keeping their projects separate from the ones that empirical statistical (and other formal and quasi-formal) methods supply. In the reverse direction, the foundational and methodological problems of these methods cannot be so readily swept aside as simply directed at a problem that is outside of those of the philosophers. In my view, this thinking has stalled progress in both arenas for the past 25 years.
Please share your comments and questions.
[i] Admittedly, there is a program of Bayesian epistemology that might be seen as doing ordinary epistemology (discussions about knowledge and beliefs) employing formal probabilities. But this is not Williamson’s project.