A reader asks how I would complete the following sentence:
I wish that new articles* written in 2014 would refrain from_______.
Here are my quick answers, in no special order:
(a) rehearsing the howlers of significance tests and other frequentist statistical methods;
(b) misinterpreting p-values, ignoring discrepancy assessments (and thus committing fallacies of rejection and non-rejection);
(c) confusing an assessment of boosts in belief (or support) in claim H ,with assessing what (if anything) has been done to ensure/increase the severity of the tests H passes;
(d) declaring that “what we really want” are posterior probability assignments in statistical hypotheses without explaining what they would mean, and why we should want them;
(e) promoting the myth that frequentist tests (and estimates) form an inconsistent hybrid of incompatible philosophies (from Fisher and Neyman-Pearson);
(f) presupposing that a relevant assessment of the scientific credentials of research would be an estimate of the percentage of null hypothesis that are “true” (selected from an “urn of nulls”) given they are rejectable with a low p-value in an “up-down” use of tests;
(g) sidestepping the main sources of pseudoscience: insevere tests through interpretational and inferential latitude, and violations of statistical model assumptions.
The “2014 wishing well” stands ready for your sentence completions.
*The question alluded to articles linked with philosophy & methodology of statistical science.
Error Statistics Philosophy 