Logic takes a bit of a hit—student driver behind me. Anyway, managed to get to JFK, and meant to explain a bit more clearly the first “shpower” post.
I’m not saying shpower is illegitimate in its own right, or that it could not have uses, only that finding that the logic for power analytic reasoning does not hold for shpower is no skin off the nose of power analytic reasoning.
Consider our one-sided test T+, with μ0= 0 and α=.025. Suppose σ = 1, n = 25, so X is statistically significant only if it exceeds .392. Suppose X just misses significance, say
X = .39. Power-analytic reasoning says (in relation to our test T+):
If X is statistically insignificant and the POW(T+, μ= μ1) is high, then X indicates, or warrants inferring (or whatever phrase you like) that μ < μ1.
Note, I am emphasizing that high power is a sufficient condition for such an inference, but that it is not a necessary condition, because the actual insignificant result can do better than just miss the cut-off for rejection. (See NN 2). I will come back to this.
So, for example, suppose X =.39, just shy of rejecting the null. POW(T+, μ1=.8) ~.98, so X =.39 indicates μ1< .8—at least by Power Analytic Reasoning.
However, a Shpower calculation with X =.39 is stuck at .5; there’s no niche for introducing μ1=.8 unless X =.8, in which case we would not be dealing with the statistically INsignificant result in the example.
So it is no wonder that there have been questions as to the relevance of high Shpower for applying Power Analysis for interpreting statistically insignificant results.
Luk Arbunkle’s 2008 post (see my Nov. 11 Post) refers, with approval, to the article by Hoenig and Heisey (2001), The American Statistician, Vol. 55, No. 1, where the same Shpower definition occurs. It is AN interesting paper, particularly if this notion is widespread. When a commentary Spanos and I wrote on the Hoenig and Heisey article was rejected some years ago, the editors said something like, it’s too late to comment, and anyway they have since found their mistakes (and perhaps retracted them????—I forget exactly). But now I’m seeing this again…..
It’s far too noisy here, and I’m afraid I’ll miss my flight.
Oh, by the way, I stripped off the knee brace and stuck it in my checked luggage this time