On August 20 I posted the start of “Discussion and Digest” by Bayesian statistician Tony O’Hagan– an oveview of responses to his letter (ISBA website) on the use of p-values in analyzing the Higgs data, prompted, in turn, by a query of subjective Bayesian Dennis Lindley. I now post the final section in which he discusses his own view. I think it raises many questions of interest both as regards this case, and more generally about statistics and science. My initial July 11 post is here.
“Higgs Boson – Digest and Discussion” By Tony O’Hagan
Discussion
So here are some of my own views on this.
There are good reasons for being cautious and demanding a very high standard of evidence before announcing something as momentous as H. It is acknowledged by those who use it that the 5-sigma standard is a fudge, though. They would surely be willing to make such an announcement if they were, for instance, 99.99% certain of H’s existence, as long as that 99.99% were rigorously justified. 5-sigma is used because they don’t feel able to quantify the probability of H rigorously. So they use the best statistical analysis that they know how to do, but because they also know there are numerous factors not taken into account by this analysis – the multiple testing, the likelihood of unrecognised or unquantified deficiencies in the data, experiment or statistics, and the possibility of other explanations – they ask for what on the face of it is an absurdly high level of significance from that analysis. Continue reading
Error Statistics Philosophy 