(Just to be clear: What is it that Senn is seemingly having little impact on (I mean in this particular post)? The tendency to suppose they know the response rates from this type of data? Sorry if I’m mistaking your referent.)

“You cannot ask us to take sides against arithmetic.” But I can ask them to take sides against a particular use of arithmetic in inference, if it fails to adequately take account of some features we would like inferences to take account of.

Anyway, I was responding to Steven McKinney.

]]>Jim did state “Bayesian analysis deals with multiplicity testing solely through the assignment of prior probabilities to models or hypotheses.” and that is a mathematical fact and to re quote Churchill “You cannot ask us to take sides against arithmetic.” But one can still get the data model wrong and or have difficulty properly interpreting posterior quantities (e.g. a Bayes Factor of 8 is stronger than 4 but is it twice as strong or strong enough to reject H0?)

Jim is taking a position on matters that are not yet widely settled where as in Stephen’s post his material does not seem at all controversial. Yes there are likely many statisticians who are unaware of these issues but most would understand them if they read Stephen’s papers and very few would disagree.

So he has been writing about this since 2001 with seemingly little impact. Perhaps there is an avoidance by other statisticians (in pharma and regulatory agencies) of being a negative nelly matched with an attraction to all the exciting highly complex statistical work that would be entailed in the glossy view of personalised medicine everywhere.

]]>Bayesians don’t believe they can magically avoid doing cumbersome adjustments. The adjustment is folded into the base rate/shrinkage/prior model rather than a multiple comparison adjustment. Matt Stephens has written about the analogy and cases when they’re equivalent in one of his reviews.

One issue I have with multiple comparison adjustments is that their effects on frequencies of errors are inconsistent as power changes, but this requires the kind of “frequency of correctness” analysis that you’re not a fan of.

]]>http://www.stat.purdue.edu/symp2012/slides/Purdue_Symposium_2012_Jim_Berger_Slides.pdf

More puzzling are the ways Bayes factors (e.g., pp. 19-20) are sold as the correct frequentist (conditional) error probabilities…This connects to a Berger paper I once commented on, and of course to my last post.

]]>Another issue, is when statisticians choose to work with life scientists they need to decide between making them happy versus enabling them to do be better science. The latter can be painful and also risky if the life scientist is paying. People like to think they can do things and often are not as critical as they should be when someone with more expertise advises them they can. Sometimes will they search for can do statistician without realising that means the statistician is naïve about the difficulties.

I know this happens, I don’t know how often, but even if it’s a minority it keeps the nonsense going. An example would be a statistician at an Ivey league medical school who confided that they stopped talking about multiplicity problems when they noticed scientists would gravitate to working with other statisticians that did not ask about that.

]]>I think the issue is numbers – there are so many more of them than us, and their culture includes derision of pedantic statisticians. How are the sensible among us to do battle with so many of them? It is not an easy pursuit, but we are making efforts. For example, the editor of the journal Science just announced the formation of a statistical board of editors to attempt to better assess the quality of statistical reasoning presented in submitted articles.

In my undergraduate and graduate studies, I observed that most people from non-statistical programs who took statistics classes really did not like them, whereas my fellow statistical colleagues loved the classes. I did not see many people at all who just kind-of liked statistics – it appears to be a love/hate relationship. Thus statisticians will always be in short supply, and we will always be noting some area or other where lack of sensible data and analysis leads people to say the silliest things based on fallacious evidence.

Referring back to the journal Science, I have lost track of how many life sciences papers I have skipped over after seeing error bars on a barplot column that extends up to 1.0 and denotes the normalized value for a control condition. Such graphics immediately indicate that the paper authors do not understand statistics appropriately, and engaged no statistician of worth in producing the analysis upon which they base their conclusions. No wonder John Ioannidis was able to produce a paper on why most published research is false.

The explosion of the biomedical field in the past two decades was not matched by an explosion in recruitment of statisticians into that field, as witnessed by the poor statistical handling of data in so many life sciences papers of this period. While it has been easy to recruit young undergraduates and graduates into life sciences, it has not, as always, been easy to recruit more statisticians into statistics programs, since so many people dislike statistics.

Professors in life sciences departments and life science researchers need to be regularly encouraged to include statistician time in their funding and grant applications, and to seek guidance from statisticians before rushing to publish. I happen to be fortunate to work for such a life scientist, but I see my situation as the exception, as few other life scientists in my geographical area have hired a statistician. Salaries in corporate jobs tend to be much higher than salaries in research jobs, so most statisticians are hoovered up into the corporate realm, where few are able to perform reasonable statistical oversight.

Thus to address this issue, top tier statisticians need to advocate heavily to organizations such as the US FDA, NCI, NIH , UK NHS etc. that grant applications must include funding for statistical support, government agencies must include statistical oversight for reports containing statistical materials, and journals must have a statistical board of editors as Science has recently set up. There’s so much evidence available now to support such arguments – so while I may not yet be ready to wag a finger at all my fellow statisticians at this point, I certainly agree that it is our responsibility to point out these problems in an appropriate tone, and advocate for better statistical support. Certainly your writings are valuable contributions towards this effort.

]]>As I say,we already have out-of-control results with financial robots: they interact with contingent facts of the world. We need more self-correction.

]]>