The answer to the question of my last post is George Barnard, and today is his 100th birthday*. The paragraphs stem from a 1981 conference in honor of his 65th birthday, published in his 1985 monograph: “A Coherent View of Statistical Inference” (Statistics, Technical Report Series, University of Waterloo). Happy Birthday George!
[I]t seems to be useful for statisticians generally to engage in retrospection at this time, because there seems now to exist an opportunity for a convergence of view on the central core of our subject. Unless such an opportunity is taken there is a danger that the powerful central stream of development of our subject may break up into smaller and smaller rivulets which may run away and disappear into the sand.
I shall be concerned with the foundations of the subject. But in case it should be thought that this means I am not here strongly concerned with practical applications, let me say right away that confusion about the foundations of the subject is responsible, in my opinion, for much of the misuse of the statistics that one meets in fields of application such as medicine, psychology, sociology, economics, and so forth. It is also responsible for the lack of use of sound statistics in the more developed areas of science and engineering. While the foundations have an interest of their own, and can, in a limited way, serve as a basis for extending statistical methods to new problems, their study is primarily justified by the need to present a coherent view of the subject when teaching it to others. One of the points I shall try to make is, that we have created difficulties for ourselves by trying to oversimplify the subject for presentation to others. It would surely have been astonishing if all the complexities of such a subtle concept as probability in its application to scientific inference could be represented in terms of only three concepts––estimates, confidence intervals, and tests of hypotheses. Yet one would get the impression that this was possible from many textbooks purporting to expound the subject. We need more complexity; and this should win us greater recognition from scientists in developed areas, who already appreciate that inference is a complex business while at the same time it should deter those working in less developed areas from thinking that all they need is a suite of computer programs.
Here’s an excerpt from the following section: “A Little History”(1)
Although I had been interested in statistics at school, in 1932, and first met Fisher in 1933, I came properly into the subject during the Second World War….[I]t was not, I think, recognized until the publication of Joan Box’s book, that the man who, more than any other, was responsible for creating the concepts now central to our subject, was cut off from these developments by some mysterious personal or political agency….
It is idle to speculate on what might have happened had the leaders of the subject, Fisher, Bartlett, Pearson, Neyman, Wald, Wilks, and others, all been engaged to work together during the war. Cynics might suggest that the resulting explosions would have made the Manhatten project redundant. But on an optimistic view we could have been spared the sharp and not particularly fruitful controversies which have beset the foundations over the past thirty years. Only now do we seem to be approaching a consensus on the respective role of “tests” or P-values, “estimates” likelihood, Bayes’ theorem, confidence or “fiducial” distributions and other more complex concepts. ….
It is interesting that Barnard calls for “more complexity” while urging “a coherent view” of statistics. I agree that a “coherent” view is possible at a foundational, philosophical level, if not on a formal level.
I’ll reblog some other posts on Barnard this week.
*There was at least one correct, original answer from Oliver Maclaren.
(1) Barnard’s rivulets remind me of Walt Whitman’s Autumn Rivulets.
I find this quote especially striking: “their study is primarily justified by the need to present a coherent view of the subject when teaching it to others.”
Fine but the devil is in the details, which vary dramatically with opinions about what is “coherent” (you and I would fail Lindley’s criterion). After that I don’t see advice that addresses today’s core problems in my fields of interest. The notion that more complexity would help does not ring true for me; rather the problem is to find what everyone would agree is both simple and correct to teach and use.
My view is that stats in soft sciences (medicine, health, social sciences among others) has been a massive educational and ergonomic failure, often self-blinded to the limits on time and capabilities of most teachers, students, and users. I suspect the reason may be that modern stats was developed and promulgated by a British elite whose students and colleagues were selected by their system to be the very best and brightest, a tiny fraction of a percent of the population. Furthermore, it was developed for fields where sequences of experiments leading to unambiguous answers could be carried out relatively quickly (over several years at most, not decades) so that the most serious errors could be detected and controlled, not left as part of the uncertainty surrounding the topic.
The statistical vision of this elite (to whom Barnard belonged) was accurate in the its time and fields. It failed as selection weakened dramatically in the explosion of university education after WWII, fueled especially by the “science race” against the Soviet bloc during the next 40 years. For example, the percent of the U.S. population going on to postgraduate training is now on the rough order of 10%; simple logic tells us that few of these could be geniuses, so no surprise that methods developed by brilliant minds to teach brilliant and well-educated students in relatively precise fields would fail as miserably in modern soft-science contexts. Stat education requires attention to the far less elite audience and far more broad applications of these times (in which to advance in scientific fields even the elite are increasingly consumed by proposal-writing and research-management responsibilities that leave ever less time for deeper thinking).