*Statistically speaking, we don’t use calculus By Dave Gammon*

An article in a local op-ed piece today (*Roanoke Times*) claims:

“Quantitative skills are highly sought after by employers, and the best time to learn these skills is in high school and early college. And we all know the best math students should eventually learn calculus.

Or should they? Maybe it’s statistics, not calculus, that is a more worthy pursuit for the vast majority of students.”

This reminds me of the trouble I got into when, as a graduate student at the University of Pennsylvania, I supplemented my fellowship in philosophy by leading some recitation classes in statistics at the Wharton school. Although it was vaguely suggested that I not assign homework problems that required calculus, since many of the exercises in the sections of the text (on business statistics) that I was to cover required, and were illuminated by, calculus, (and given that the text was written by a Wharton statistics professor [de Cani]), I went ahead and assigned some of them, and promptly was reported by the students[i]. The author of this article appears to have no clue that statistical methods depend on calculus and the “area under a curve”.

It’s not that learning calculus is pointless. Calculus applies integrally to physics, engineering, space exploration, satellite positioning, complex building projects and several other lines of work. If your career ambitions fit within one of these domains, by all means, learn calculus. Most of us, however, are destined for careers other than physics, engineering or mathematics.

Statistics weave deeply throughout the fabric of our society. Remember all those political polls last fall? Every one of them reported a “margin of error” and a “sample size” as part of the results. Without statistical training, it is easy to misinterpret their results — especially when you encounter conflicting poll results….

Because of the news media and the Internet, we face a daily bombardment of quantitative information. This deluge is growing exponentially with each passing year and each technological innovation, becoming a virtual river into which our vessels are thrust. A citizen without a working knowledge of statistics is rafting that river without a paddle.

Can you remember the last time you saw in the newspaper a story that focused on the area under a curve, or the slope of a line? Yeah, neither can I. If you ever stumble upon such a story, then calculus might be your best friend.

To most students, however, statistics will be of far more use than calculus, especially if they select career paths that require them to analyze risk, evaluate trends, confront randomness, deal with uncertainty or extract patterns from data…

Not once in my professional career has calculus been particularly relevant.

Statistics, however, has been my bread and butter. Nearly every major project I have attempted required the use of statistics. I wish someone had told me this secret when I was younger……

Ultimately, it will take time for our educational systems to adapt to the shifting needs of society. Students will benefit right now, however, if they expose themselves to more statistics. Parents and academic advisers everywhere should take heed and help promote this valuable discipline.

Admittedly, a lot of the statistics we read about remains stuck at a shallow level; and perhaps the “virtual river into which” this author’s vessel is thrust will never have to scratch far below the surface. Else, he might have to discover that statistical methods are based on areas under curves.

[i] I made up “recipe sheets” for the rest of the semester.

Can you recommend articles, websites or textbooks that illuminate statistics using calculus? I’ve been looking for more “proof-like” explanations of statistics for a while.

This was my bible when I was learning the mathematics behind stats:

http://probability.ca/jeff/probstatbook.html

From the preface:

“This book is an introductory text on probability and statistics. The book is targeted at students who have studied one year of calculus at the university level and are seeking an introduction to probability and statistics that has mathematical content. Where possible, we provide mathematical details, and it is expected that students are seeking to gain some mastery over these, as well as learn how to conduct data analyses. All of the usual methodologies covered in a typical introductory course are introduced, as well as some of the theory that serves as their justification. ”

From 2004 so could be too old now?

Would nine years really make it too old? Presumably the proofs for most of the statistics wouldn’t have changed…

Its more the choice of material that I’m thinking about, than the proofs themselves. For example, in recent years people have really lost interest in classical topics such as minimum variance unbiased estimation, and as I remember these are covered in this book. We don’t want too much bias in our estimates, but demanding that they be ‘minimum variance’ is a little much given the complexity of modern problems.

Still, its a great book. Don’t get me wrong.

But it will still be good to know the mini variance UB estimator,no?

I used Mood, Graybill and Boes, Introduction to the Theory of Statistics, also at Univ. Penn. Long story as to how I accidentally wound up in a stat class.

Annoyingly, this sort of stupidity posing as middle-brow contrarian-ness seems to be catching on. Even worse is this New York Times editorial suggesting that students learn “citizen statistics” in place of algebra http://www.nytimes.com/2012/07/29/opinion/sunday/is-algebra-necessary.html?pagewanted=all

rv: You’re right, your article is even worse, especially in the NYT, though not so surprising. Advocating dumbing down “as middle-brow contrarian-ness” indeed. Welcome to the vocational math society. D

Surprised you are not keener on this; the emphasis is on getting a general audience to embrace quantitative reasoning, and to think critically about what data can reasonably tell us, and what it can’t.

More technically, statisticians often get “areas under the curve” by methods very far from high school calculus – though not beyond the comprehension of (good) high school students. Examples include permutation tests, and using the bootstrap. Teaching these ideas instead of e.g. trigonometry seems worth looking at.

OG: Unfortunately, there is no evidence that popularizing these topics enhances thinking critically about “what data can reasonably tell us, and what it can’t”, but rather the opposite. To take just one example, there is the common tendency to suppose that statistical significance tests permit or even encourage a host of fallacies based on misinterpretations that the tests were actually designed to avoid (e.g., ignoring multiple testing, data-dependent end points, invalid underlying statistical models, and much more). There is lack of awareness as to how sample size relates to discrepancies detected, and confusion as to how to even define p-values, power, etc. and how these error probabilities relate to posterior probabilities. Even the so-called “reformers” these days get many of the notions backward. You can search this blog for many examples. (That is why I am trying to write a book.)

DM: I am well aware of the issues of authors/textbooks mis-representing statistical issues, in part through being a longtime reader and contributor here.

I disagree about “rather the opposite”. While there are a bunch of poor courses and inept statisticians – i.e. the ones you criticize here – in my experience there are also many good ones. Some of the good courses are aimed at high school level, and require no calculus.

I did have in mind college courses. Calculus is taken by many/most(?) high school students (which I do not think should dropped), but a high school statistics class need not involve calculus. My son had a high school course using Peck, Olsen, and Devore which impressed me. Not only did it consistently interpret methods correctly, it included testing statistical assumptions.

There are two ways to understand statistical concepts and methods. One involves calculus, and the other avoids all mathematics beyond arithmetic. The first is a good one, but it is accessible to a tiny fraction of my students.

It is my experience that the mathematically inclined (or sophisticated) are slow to understand that there are other approaches that can lead to practical mastery of statistics.

The philosophy of statistics is probably as important as the mathematical proofs, and it is even more rarely talked about in introductory statistics courses.

I try to help my students understand statistics and to have a reasonably reliable intuition: the proofs are of little use to most of them. I am inclined to agree that statistical thinking is more important than calculus-ical thinking for most students.

Michael: I might agree that “The philosophy of statistics is probably as important as the mathematical proofs” except for the fact that there is considerable confusion and disagreement today about underlying philosophies. Thus it is good to give people tools to determine how to tell what’s true about statistical inference, and what is not. “Philosophy laden” teaching of statistics is highly problematic, as I see it–especially as the very fact that it is “philosophy laden” tends to be hidden.

It was not a fun subject. But I did like Algebra. Haven’t used functions much or quadratic equations in the last few weeks.

MHM: what would Mr. Edge say?

Statistics counts whereas calculus can be very derivative and analysis is the limit. 🙂 However, the interesting issue for me is what level of calculus is necessary to do statistics? I use calculus explicitly, if not daily then weekly as part of my statistical work and in a sense implicitly all the time but that doesn’t mean that I ‘know’ calculus. A controversy, for example, is the place of measure theory: essential to some and a misleading blind alley to others.

Stephen: You’re right that the level counts; I thought the students at Wharton should not have been “deprived” of using the intro level of calculus called for in the exercises.