We are pleased to announce our guest speaker at Thursday’s seminar (April 24, 2014): “Statistics and Scientific Integrity”:
S. Stanley Young, PhD
Assistant Director for Bioinformatics
National Institute of Statistical Sciences
Research Triangle Park, NC
Author of Resampling-Based Multiple Testing, Westfall and Young (1993) Wiley.
The main readings for the discussion are:
- Young, S. & Karr, A. (2011). Deming, Data and Observational Studies. Signif. 8 (3), 116–120.
- Begley & Ellis (2012) Raise standards for preclinical cancer research. Nature 483: 531-533.
- Ioannidis (2005). Why most published research findings are false. PLoS Med 2(8): e124.
- Peng, R. D., Dominici, F. & Zeger, S. L. (2006). “Reproducible Epidemiologic Research” American Journal of Epidemiology 163 (9), 783-789.
Error Statistics Philosophy 
Looks very interesting. I hope we can get a copy of the presentation.
Stan Young led a great seminar this afternoon. He had us toss 10-sided dice to demonstrate the ease of generating statistical significance by chance–if you allow multiple testing and report just the stat sig ones. Spurious p-values, in short. I’ll post his slides tomorrow.
Maybe this was covered in the lecture, but doesn’t this dice demonstration miss the point about base rates bought up in the Ioannidis paper (and elsewhere)?
In my opinion, multiple comparisons also tend to get overemphasized in bioinformatics to the detriment of other issues such as power considerations and effect overestimation.
In many bioinformatics applications such as gwas and differential expression analysis, multiple comparison adjustments might be considered a roundabout way to account for low base rates without actually modeling them.
There are lots of ways that an experiment can come to grief, fail to replicate. Asking lots of questions and not taking multiple comparisons into account is just one problem. Dealing with multiple questions is largely a solved technical problem, Westfall and Young, 1993. Independent replication is the experimental way forward.