RSS 2018 – Significance Tests: Rethinking the Controversy


Day 2, Wednesday 05/09/2018

11:20 – 13:20

Keynote 4 – Significance Tests: Rethinking the Controversy Assembly Room

Sir David Cox, Nuffield College, Oxford
Deborah Mayo, Virginia Tech
Richard Morey, Cardiff University
Aris Spanos, Virginia Tech

Intermingled in today’s statistical controversies are some long-standing, but unresolved, disagreements on the nature and principles of statistical methods and the roles for probability in statistical inference and modelling. In reaction to the so-called “replication crisis” in the sciences, some reformers suggest significance tests as a major culprit. To understand the ramifications of the proposed reforms, there is a pressing need for a deeper understanding of the source of the problems in the sciences and a balanced critique of the alternative methods being proposed to supplant significance tests. In this session speakers offer perspectives on significance tests from statistical science, econometrics, experimental psychology and philosophy of science. There will be also be panel discussion.

Categories: Error Statistics | 2 Comments

Post navigation

2 thoughts on “RSS 2018 – Significance Tests: Rethinking the Controversy

  1. Huw Llewelyn

    Thank you for your helpful preview. I look forward very much to attending this session. I note that there is no perspective from a medical diagnosis point of view. I do this by comparing the prior probabilities of diagnostic and statistical inferences in a talk on the first day at 9.20am on Tuesday: ‘Teaching the concept of prior probability to medical students, doctors and other non-statisticians using ‘P maps’.

    This is the abstract:

    There are two types of prior probability: Unconditional ‘base rate’ (but conditional on the feature(s) defining a universal set) and a conditional ‘non base rate’ prior probability conditional on the features of a universal set and at least one of its subsets. The difference can be clarified during teaching by using ‘P maps’ in conjunction with Venn diagrams[1]. The probability of the defining features of a universal set conditional on any of its subset features is ‘one’ but this is not the case for non base rate priors. P maps can clarify this and the nature of prior probabilities in clinical practice and how they can be used with and without the assumption of statistical independence. This can be compared with the role of priors when estimating parameters by using random sampling. P maps can be used to show how P values can be used to estimate the probability of ‘idealistic’ replication (when a study is described impeccably by an author and repeated impeccably by a reader) and how prior information can be incorporated into the evidence to estimate the probability of ‘realistic’ replication. They can also be used to represent how a single finding is linked to a list of possible diagnoses, each diagnosis having a conditional non-base-rate prior probability. The probabilities of the latter can be updated with new evidence by multiplying the ratios of a pair of prior probabilities (one of which belongs to the postulated diagnosis) with the ratio of a pair of corresponding likelihoods using a derivation of the extended form of Bayes rule. Examples will be provided using data from a study on the differential diagnosis of acute abdominal pain.

    1 Llewelyn H, Ang AH, Lewis K, Abdullah A. (2014) Picturing probabilities. In The Oxford Handbook of Clinical Diagnosis, 3rd edition. Oxford University Press, Oxford. p 618-621.

  2. psychnstatstutor Chart your course to success

    Absolutely, especially as students are learning NHST and effect sizes remain a slight mention in class. Students will report them in their research reports, however they rarely comment on them because they don’t understand their point.

I welcome constructive comments. If you want to correct or remove a comment, send me an e-mail. If readers have already replied to the comment, you may be asked to replace it to retain comprehension.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at