SEV calculator (with comparisons to p-values, power, CIs)
In the illustration in the Jan. 2 post,
H0: μ < 0 vs H1: μ > 0
and the standard deviation SD = 1, n = 25, so σx = SD/√n = .2
Setting α to .025, the cut-off for rejection is .39. (can round to .4).
Let the observed mean X = .2 , a statistically insignificant result (p value = .16)
SEV (μ < .2) = .5
SEV(μ <.3) = .7
SEV(μ <.4) = .84
SEV(μ <.5) = .93
SEV(μ <.6*) = .975
*rounding
Some students asked about crunching some of the numbers, so here’s a rather rickety old SEV calculator*. It is limited, rather scruffy-looking (nothing like the pretty visuals others post) but it is very useful. It also shows the Normal curves, how shaded areas change with changed hypothetical alternatives, and gives contrasts with confidence intervals.
It is designed for H0: μ < 12 vs H1: μ > 12 (and values in a small range).**
To get corresponding calculations to those with H0: μ < 0 write in 12 for the population mean.
Turquoise box
In the turquoise box, you see population SD, let it be 1
Below that, sample size n. To accord with this illustration, let n = 25, so σx = SD/√n = .2.
Setting α to .025, the cut-off for rejection is 12.39.
Illustration #1: consider outcomes that do not reach the cut-off for statistical significance, so the test hypothesis is not rejected (ie., in Neyman’s shorthand, it is “accepted”.)
Grey box
In the grey box, let the sample mean (X) be a non-significant value.
To illustrate, set X to 12.2 in the grey box. (You have to click outside the box you write in to get it to register.)
The grey box also shows the p value.
Orange box
In the orange box below you can set different values for the population mean in the alternative hypothesis space: μ > 12 + γσx
(You click on the box, write in the value, then you have to click to get it to update)
The severity value is shown to the right.
Let X = 12 .2 and the SD = 1, n = 25, so again σx = SD/√n = .2.
Interest is in considering claims of form: μ < 12 + γσx
SEV (μ < 12.2) = .5
SEV(μ < 12.3) = .7
SEV(μ <12.4) = .84
SEV(μ <12.5) = .93
SEV(μ <12.6*) = .975
*rounding
Green box: shows power
Next try n=100 keeping everything else the same. (What do you get?)
NOTE: The actual definition of SEV(μ < μ1) = P(d(X) > d(x0); μ < μ1 false) = P(d(X) > d(x0); μ > μ1)
As with power, severity is evaluated at a point μ1= (μ0 + γ), for some γ > 0; yet the above holds because for values μ > μ1 the severity increases. The above is just a rule of thumb for getting Sev in this case. Spanos, rightly, thinks one should go back to the definition to get the answers, since the quick form appears to mix parameter and sample space. But they come out the same.
*It was created by Geoff Cumming in like one afternoon, at our “break out group” at NCEAS (Nat Cntr for Ecol Anal and Synthesis) several years ago.
**The Sev calculator corresponds to the example used in EGEK (Mayo 1996), and throughout the following paper:
Mayo, D. G. and Spanos, A. (2006). “Severe Testing as a Basic Concept in a Neyman-Pearson Philosophy of Induction,” British Journal of Philosophy of Science, 57: 323-357.
Error Statistics Philosophy 