A commentator brings up risk factor epidemiology, and while I’m not sure the following very short commentary* by Aris Spanos and I directly deals with his query, Greenland happens to mention Popper, and it might be of interest: “**When Can Risk-Factor Epidemiology Provide Reliable Tests?**”

Here’s the abstract:

Can we obtain interesting and valuable knowledge from observed associations of the sort described by Greenland and colleagues in their paper on risk factor epidemiology? Greenland argues “yes,” and we agree. However, the really important and difficult questions are when and why. Answering these questions demands a clear understanding of the problems involved when going from observed associations of risk factors to causal hypotheses that account for them. Two main problems are that 1) the observed associations could fail to be genuine; and 2) even if they are genuine, there are many competing causal inferences that can account for them. Although Greenland’s focus is on the latter, both are equally important, and progress here hinges on disentangling the two to a much greater extent than is typically recognized.

* We were commenting on “**The Value of Risk-Factor (“Black-Box”) Epidemiology**” by Greenland, Sander; Gago-Dominguez, Manuela; Castelao, Jose Esteban full citation & abstract can be found at the link above.

Thanks for this, Deborah, Greenland is another of my statistical heroes! I agree that valuable information can be obtained from observed associations, but I suppose that depends on what use an inferential approach would make of them. Greenland et al. seem to be arguing more for a descriptive “qualitative” inference based on assessing whether or not and which observed associations are consistent with any proposed hypothesis. This seems to be the same approach that Gary Taubes took in Good Calories, Bad Calories and it’s an approach that I find particularly convincing (although I understand that Taubes is sometimes accused of cherry-picking evidence, I don’t know enough about the nutrition literature to evaluate whether this is true). And it is certainly consistent with your proposed piecemeal approach.

However, in his response, and true to his fabulous 1990 paper in Epidemiology, Greenland seems to shun the error-statistical approach to evaluating observed associations: “But this key [i.e., ‘to establish the validity of statistical assumptions without appeals to substantive theories’] is missing in observational epidemiology. Its absence *pulls the plug on frequentist methods*, for those assume the data arise from experiments with adequate control or knowledge of errors (both random and systematic)… hence, there are enduring controversies about whether certain associations even exist, let alone are causal.” I think I agree with Greenland here.

Mark, but of course it’s incorrect to allege that error statistical methods only apply in “genuine” statistical experiments. It suffices that statistical models can be used to capture some question or aspect of the process generating the data. The methods are very often used in historical and observational studies; else those areas would be robbed of statistical inquiries. There are “model based” as well as “design based” uses of these methods, for lack of better terms.