Memento & Quiz (on SEV): Excursion 3, Tour I

.

As you enjoy the weekend discussion & concert in the Captain’s Central Limit Library & Lounge, your Tour Guide has prepared a brief overview of Excursion 3 Tour I, and a short (semi-severe) quiz on severity, based on exhibit (i).*

 

We move from Popper through a gallery on “Data Analysis in the 1919 Eclipse tests of the General Theory of Relativity (GTR)” (3.1) which leads to the main gallery on the origin of statistical tests (3.2) by way of a look at where the main members of our statistical cast are in 1919: Fisher, Neyman and Pearson. From the GTR episode, we identify the key elements of a statistical test–the steps in E.S. Pearson’s opening description of tests in 3.2. The classical testing notions–type I and II errors, power, consistent tests–are shown to grow out of requiring probative tests. The typical (behavioristic) formulation of N-P tests came later. The severe tester breaks out of the behavioristic prison. A first look at the severity construal of N-P tests is in Exhibit (i). Viewing statistical inference as severe testing shows how to do all N-P tests do (and more) while a member of the Fisherian Tribe (3.3). We consider the frequentist principle of evidence FEV and the divergent interpretations that are called for by Cox’s taxonomy of null hypotheses. The last member of the taxonomy–substantively based null hypotheses–returns us to the opening episode of GTR.

key terms (incomplete please send me yours)

GTR, eclipse test, ether effect, corona effect, PPN framework, statistical test ingredients, Anglo-Polish collaboration, Lambda criterion; Type I error, Type II error, power, P-value, unbiased tests, consistent tests uniformly most powerful (UMP); severity interpretation of tests, severity function, water plant accident; sufficient statistic; frequentist principle of evidence FEV; sensitivity achieved, [same as attained power (att power)], Cox’s taxonomy (embedded, nested, dividing, testing assumptions), Nordvedt effect, equivalence principle (strong and weak)

Semi-Severe Severity Quiz, based on the example in Exhibit (i) of Excursion 3

  1. Keeping to Test T+ with H0: μ ≤ 150 vs. H1: μ > 150, σ = 10, and n = 100, observed x  = 152 (i.e., d = 2), find the severity associated with μ > 150.5 .

i.e.,SEV100(μ > 150.5) = ________

  1. Compute 3 or more of the severity assessments for Table 3.2, with x  = 153.
  2. Comparing n = 100 with n = 10,000: Keeping to Test T+ with H0: μ ≤ 150 vs. H1: μ > 150, σ = 10, change the sample size so that  n = 10,000.

The 2SE rejection rule would now be: reject (i.e., “infer evidence against H0”) whenever X   > _____.

Assume x  = just reaches this 2SE cut-off. (added previous sentence, Dec 10, I thought it was clear.) What’s the severity associated with inferring μ > 150.5 now?

i.e., SEV10,000(μ > 150.5) = ____

Compare with SEV100(μ > 150.5).

4. NEW. I realized I needed to include a “negative” result. Assume x  = 151.5. Keeping to the same test with n = 100, find SEV100(μ ≤ 152).

5. If you’re following the original schedule, you’ll have read Tour II of Excursion 3, so here’s an easy question: Why does Souvenir M tell you to “relax”?

6. Extra Credit: supply some key terms from this Tour that I left out in the above list.

*The reference is to Mayo (2018, CUP): Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars.

 

Categories: Severity, Statistical Inference as Severe Testing | 8 Comments

Post navigation

8 thoughts on “Memento & Quiz (on SEV): Excursion 3, Tour I

  1. 1. SEV(μ > 150.5) is the worst case (i.e., highest) probability, under μ ≤ 150, of getting a test statistic that accords less with “μ > 150.5” than does the observed test statistic x_bar = 152. So,

    SEV(μ > 150.5) = Pr(X_bar ≤ 152; μ = 150) = 0.93

    2. nah

    3. In Q1 the standard error was 1; now the standard error is 0.1 and the 2SE rejection threshold is 150.2 and SEV(μ > 150.5) = 1 – tiny ε

    4. Back to SE = 1. SEV(μ ≤ 152) is the worst case (i.e., highest) probability, under μ > 152, of getting a test statistic that accords less with “μ ≤ 152” than does the observed test statistic x_bar = 151.5. So,

    SEV(μ ≤ 152) = Pr(X_bar > 151.5; μ = 152) = 0.69

    5. I’m not there yet. Let’s see… probably something to do with how severity clears up confusions and makes equivocations that paper over disagreements between various tribes unnecessary.

    So I have a question — a genuine question, mind, not a trick question or a rhetorical point-making one — regarding the application of severity reasoning in discrete settings. Suppose I have a measurement device that answers some binary yes/no question. The device is known to give correct answers 7 times in 10 and to give answers selected uniformly at random 3 times in 10.

    The severity with which H passes a test with some test statistic, we are told, is the (worst-case, but that’s not relevant here) probability of getting a test statistic that accords less well with H than the one observed, supposing H to be false.

    I have yet to see how to measure accordance in the discrete setting (perhaps it’s described in sections of the book I haven’t read yet). The question is: does accordance encompass being correct by happy chance in the 3 times in 10 that the measurement device gives a random answer? If so, severity is the proportion of times the answer is correct, 17 times in 20. But I suspect not, so I guess my real question is: is severity just the proportion of times the measurement device operates correctly?

    • MAYO GRADES COREY’S QUIZ.

      1. SEV(μ > 150.5) = Pr(X_bar ≤ 152; μ = 150) = 0.93
      MAYO: Correct numerical answer, but it should be:
      SEV(μ > 150.5) = Pr(X_bar ≤ 152; μ = 150.5) = 0.93. Half off.

      2. nah

      3. In Q1 the standard error was 1; now the standard error is 0.1 and the 2SE rejection threshold is 150.2 and SEV(μ > 150.5) = 1 – tiny ε

      MAYO: Incorrect. I should have added “show your work”,then you’d spot errors. However, it’s possible you read the question assuming the observed mean M0 = 152, whereas it was to be the 2SE cut-off. I added a sentence.

      4. Back to SE = 1. SEV(μ ≤ 152) is the worst case (i.e., highest) probability, under μ > 152, of getting a test statistic that accords less with “μ ≤ 152” than does the observed test statistic x_bar = 151.5. So,

      SEV(μ ≤ 152) = Pr(X_bar > 151.5; μ = 152) = 0.69
      MAYO: Yes.

      5. I’m not there yet. Let’s see… probably something to do with how severity clears up confusions and makes equivocations that paper over disagreements between various tribes unnecessary.
      MAYO: Not even close.

      MAYO: So it appears that you got 1 1/2 out of 5. You may wish to resubmit.

  2. I misread Q1 at first; when I noticed I fixed the answer but missed the typo in the SEV expression.

    For the normal model 2SE is always SEV = 0.98; that’s what I had for Q1 before I noticed that I used the wrong mean.

    Curious that you failed to criticizing me for writing that the worst case severity is the highest probability instead of the lowest. I originally started writing my answers using (S-2)* instead of (S-2) and I missed that correction too.

    • Corey’s response to my grading his SEV quiz reminds me of some students I’ve had. And when one is grading 30 papers with circuitous stories about why they thought this or that and read it wrong but knew the right ans, and why didn’t I mark off for something else that was confused or backwards, one tries to give the most generous reading.
      But I know Corey knows these computations. I spoze he just tried doing them standing up while doing something else because they are so easy. If you send me a revised quiz, I will happily remove your first attempt.

      Attempting a quiz or other assignment earns reward cruise pts. 7 pts for a prize.

  3. Keeping to those I could do on my phone + WolframAlpha I got:

    1. Claim is mu > 150.5 in light of xbar = 152

    Hence compute prob disagree under not mu = 1-agree as

    1- max P(x > 152 ; mu less than or equal 150.5, sigma = 1)
    =
    0.933

    2. Don’t have book on me right now, will do later.

    3. As in 1 but now compute

    1- max P(x > 150.2 ; mu less than or equal 150.5, sigma = 0.1)
    =
    0.17

    4. Compute 1- max P (x less than or equal 151.5; mu >= 152)
    =
    0.69

I welcome constructive comments for 14-21 days. If you wish to have a comment of yours removed during that time, send me an e-mail. If readers have already replied to the comment, you may be asked to replace it to retain comprehension.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.