Higgs Boson: Bayesian “Digest and Discussion”

Professor  O’Hagan sent around (to the ISBA list ) his summary of the comments he received in response to his request for information about the use of p-values in in relation to the Higgs boson data. My original July 11 post including O’Hagan’s initial letter is here.  His “digest” begins:

Before going further, I should say that the wording of this message, including the somewhat inflammatory nature of some parts of it, was mine; I was not quoting Dennis Lindley directly. The wording was, though, quite deliberately intended to provoke discussion. In that objective it was successful – I received more than 30 substantive comments in reply. All of these were thoughtful and I learnt a great deal from them. I promised to construct a digest of the discussion. This document is that digest and a bit more – it includes some personal reflections on the issues.

In what follows, particular contributors are occasionally named wherever the context seemed to require, but on the whole I have not attempted to give credit for each piece of information or each viewpoint. A list of contributors is given at the end of this document and most of their contributions can be found on the ISBA website at http://bayesian.org/forums/news/3648.

Has anyone tried to explain?

Probably my most inflammatory comments were in question 2: Are the particle physics community completely wedded to frequentist analysis? If so, has anyone tried to explain what bad science that is?

I already knew of several serious physicists who had been propounding Bayesian ideas within the physics community, two of whom (Bill Jefferys and Giulio d’Agostini) responded to my message. In particular, d’Agostini’s article http://arxiv.org/abs/1112.3620 is specifically relevant to the Higgs boson issue.

More to the point, several respondents told me about the Phystat meetings organised by Louis Lyons. The recent one in 2011 was mostly devoted to the very question of ‘discovery’ and the 5- sigma rule. Several eminent statisticians made presentations. In its programme (http://indico.cern.ch/conferenceOtherViews.py?view=standard&confId=107747) I recognised David Cox, Jim Berger, José Bernardo and David van Dyk. I understand that other statisticians, including Michael Goldstein, Steffen Lauritzen and Persi Diaconis, had been involved in earlier meetings. It is important to say that Phystat may not be a mainstream physics meeting, but it does demonstrate that at least a subset of physicists do engage with statisticians on questions of philosophy and methodology. A couple of weeks after posting my message, I attended the 2012 Maxent meeting (http://www.ipp.mpg.de/ippcms/eng/for/veranstaltungen/konferenzen/maxent2012/). This is a long-standing series of annual conferences which, although it has a maximum entropy focus, also takes Bayesian ideas seriously.

Broadly speaking, it seems that Bayesian methods are prominent in the astrophysics end of the physics community, but at the other end the particle physicists are, if not ‘completely wedded’, certainly mostly committed to frequentist methods. But it is not because nobody has attempted to convert them to Bayesian thinking, nor would it be at all fair to say that no particle physicists have listened to those arguments. Inevitably, I am sure the great majority of particle physicists do not concern themselves with the foundations of statistics, and simply follow the prevailing view; they were taught frequentist methods at university, all of their colleagues use them and the journals expect them. Nevertheless there are some who do think about such things, and in the particle physics community it seems that the overwhelming majority favour frequentist methods.

Specifically in regard to the Higgs boson, I read that “There is a bureaucracy that determines in advance the procedures (statistical and otherwise) that will be implemented. Like most bureaucracies, this one is rather conservative and is unlikely to be far out-of-step with the general consensus of its scientific community, a community that is much larger than group that participates in Phystat workshops. So far, the official statistical methods have not been Bayesian.” Whether this bureaucracy is drawn from those who are not concerned about the foundations of statistics and just follow precedent, or whether some of them have given serious thought to the issue is not mentioned.

Particle physicists are exploring the fundamental nature of matter and generally regard science as uncompromisingly objective. I believe that this is misguided, but the claimed objectivity of science is deeply ingrained. Regardless of this particular point, one physicist remarked that “most find personal Bayesianism decidedly uncongenial as a way of proceeding with a publication with hundreds (actually thousands) of signatories.” For an announcement as momentous as the discovery of the Higgs boson, we are not concerned with the personal judgements of individual scientists. Whilst it may be feasible (and from a Bayesian perspective, the right thing to do) to evaluate the individual physicists’ posterior beliefs and assess their degree of consensus, such consensus would surely require very compelling evidence.

It remains to consider just how compelling the evidence is, and in this regard the meaning of 5-sigma is obviously a key factor.

To read his entire “digest”:  http://tonyohagan.co.uk/academic/pdf/HiggsBoson.pdf.

Please share your comments once you digest the ‘digest”.