3 YEARS AGO (JANUARY 2014): MEMORY LANE

One thought on “3 YEARS AGO (JANUARY 2014): MEMORY LANE”

1. 

   February 1, 2017
   lauriedavies2014

   Jeffreys on p-values.

   Suppose the hypothesis is that the data follow the $N(0,1)$
   distribution. What observable results does this hypothesis predict? It
   seem pointless to predict a single value as such a prediction would be
   wrong with probability 1. The prediction must be a set ${\mathcal S}$
   of values with the prediction being regarded as correct if the
   observable result $x$ lies in ${\mathcal S}$. Putting ${\mathcal
   S}=\rz$ results in the prediction being correct with probability one
   but this is somewhat vacuous. A non-vacuous prediction can be obtained
   by specifying a probability $\alpha$ and a set ${\mathcal S}(\alpha)$
   such that the prediction is correct with probability $\alpha$, $\pr(X\in
   {\mathcal S}(\alpha))=\alpha$. It is worthy of note that the larger
   $\alpha$ the more vacuous the prediction so to speak. As a simple example put
   $\alpha=0.95$ and ${\mathcal S}(\alpha)=(-1.96,1.96)$ and suppose that
   $x=3.121$ is observed. The $p$-value is $\pr(\vert X \vert > 3.121)=
   0.0018$ and for this to be a successful prediction would require
   $\alpha=0.9982$ rather than the chosen $\alpha=0.95$. We now interpret
   `not predicted to occur’ in the sense `predicted not to
   occur’ rather than in the sense `forgetting to predict’. If it were
   agreed beforehand that a false prediction would lead to the null
   hypothesis to be rejected, then this is done because a value predicted
   not to occur, namely 3.121, did in fact occur. This seems an
   unremarkable procedure. How bad the prediction error is can be
   measured by the $\alpha=0.9982$ required to make the prediction
   correct and which corresponds to a very weak prediction in that it
   would be correct in 99.8\% of the times.