Excursion 3 Tour III:
A long-standing family feud among frequentists is between hypotheses tests and confidence intervals (CIs). In fact there’s a clear duality between the two: the parameter values within the (1 – α) CI are those that are not rejectable by the corresponding test at level α. (3.7) illuminates both CIs and severity by means of this duality. A key idea is arguing from the capabilities of methods to what may be inferred. CIs thereby obtain an inferential rationale (beyond performance), and several benchmarks are reported.
In (3.8) we reopen a highly controversial matter of interpretation in relation to statistics and the 2012 discovery of the Higgs particle based on a “5 sigma observed effect”. Because the 5 sigma standard refers to frequentist significance testing, the discovery was immediately imbued with controversies that, at bottom, concern statistical philosophy. Some Bayesians even hinted it was “bad science”. One of the knottiest criticisms concerns the very meaning of the phrase: “the probability our results are (merely) a statistical fluctuation”. Failing to clarify it may impinge on the nature of future big science inquiry. The problem is a bit delicate, and my solution is likely to be provocative. Even rejecting my construal will allow readers to see what it’s like to switch from wearing probabilist, to severe testing, glasses.
Confidence intervals, lower bound, upper bound
Confidence intervals, duality with tests
Duality between CI inferences and severity
Capability and severity
Rubbing off interpretation
Confidence distributions (CD)
Confidence level (coefficient)
Meaning vs application gap (in interpreting CIs)
Higg’s particle
ISBA (International Society for Bayesian Analysis)
Look elsewhere effect (local and global P-values)
5 sigma
P-value police
Beyond standard model physics (BSM)
Probable flukes
ASA P-value guide
Chestnuts:
A 95% CI known to be true
Confidence sets might have high overall coverage while known to be true in given cases
rigged and pathological CIs
Excerpts from Excursion 3 Tour III are here
Where you are in the journey.
Error Statistics Philosophy 