By: Nathan Schachtman, Esq., PC*
When the Supreme Court decided this case, I knew that some people would try to claim that it was a decision about the irrelevance or unimportance of statistical significance in assessing epidemiologic data. Indeed, the defense lawyers invited this interpretation by trying to connect materiality with causation. Having rejected that connection, the Supreme Court’s holding could address only materiality because causation was never at issue. It is a fundamental mistake to include undecided, immaterial facts as part of a court’s holding or the ratio decidendi of its opinion.
Interstitial Doubts About the Matrixx
Statistics professors are excited that the United States Supreme Court issued an opinion that ostensibly addressed statistical significance. One such example of the excitement is an article, in press, by Joseph B. Kadane, Professor in the Department of Statistics, in Carnegie Mellon University, Pittsburgh, Pennsylvania. See Joseph B. Kadane, “Matrixx v. Siracusano: what do courts mean by ‘statistical significance’?” 11[x] Law, Probability and Risk 1 (2011).
Professor Kadane makes the sensible point that the allegations of adverse events did not admit of an analysis that would imply statistical significance or its absence. Id. at 5. See Schachtman, “The Matrixx – A Comedy of Errors” (April 6, 2011)”; David Kaye, ” Trapped in the Matrixx: The U.S. Supreme Court and the Need for Statistical Significance,” BNA Product Safety and Liability Reporter 1007 (Sept. 12, 2011). Unfortunately, the excitement has obscured Professor Kadane’s interpretation of the Court’s holding, and has led him astray in assessing the importance of the case.
In the opening paragraph of his paper, Professor Kadane quotes from the Supreme Court’s opinion that “the premise that statistical significance is the only reliable indication of causation … is flawed,” Matrixx Initiatives, Inc. v. Siracusano, U.S. , 131 S.Ct. 1309 (2011). The quote is accurate, but Professor Kadane proceeds to claim that this quote represents the holding of the Court. Kadane, supra at 1. The Court held no such thing.
Matrixx was a security fraud class action suit, brought by investors who claimed that the company misled them when they spoke to the market about the strong growth prospects of the company’s product, Zicam cold remedy, when they had information that raised concerns that might affect the product’s economic viability and its FDA licensure. The only causation required for the plaintiffs to show was an economic loss caused by management’s intentional withholding of “material” information that should have been disclosed under all the facts and circumstances. Plaintiffs do not have to prove that the medication causes the harm alleged in personal injury actions. Indeed, it might turn out to be indisputable that the medication does not cause the alleged harm, but earlier, suggestive studies would provoke regulatory intervention and even a regulatory decision to withdraw the product from the market. Investors obviously could be hurt under this scenario as much as, if not more than, if the medication caused the harms alleged by personal-injury plaintiffs.
Kadane’s assessment goes awry in suggesting that the Supreme Court issued a holding about facts that were neither proven nor necessary for it to reach its decision. Courts can, and do, comment, note, and opine about many unnecessary facts or allegations in reaching a holding, but these statements are obiter dicta, if they are not necessary to the disposition of the case. Because medical causation was not required for the Supreme Court to reach its decision, its presence or absence was not, and could not, be part of the Court’s holding.
Kadane makes a similar erroneous statement that the lower appellate courts, which earlier had addressed “statistical significance,” properly or improperly understood, found that “statistical significance in the strict sense [was] neither necessary … nor sufficient … to require action to remove a drug from the market.” Id. at 6. The earlier appellate decisions addressed securities fraud, however, not regulatory action of withdrawal of a product. Kadane’s statement mistakes what was at issue, and what was decided, in all the cases discussed.
Kadane seems at least implicitly to recognize that medical causation is not at issue when he states that “the FDA does not require proof of causation but rather reasonable evidence of an association before a warning is issued.” Id. at 7 (internal citation omitted). All that had to have happened for the investors to have been harmed by the Company’s misleading statements was for Matrixx Initiatives to boast about future sales, and to claim that there were no health issues that would lead to regulatory intervention, when they had information raising doubts about their claim of no health issues. See FDA Regulations, 21 U.S.C. § 355(d), (e)(requiring drug sponsor to show adequate testing, labeling, safety, and efficacy); see also 21 C.F.R. § 201.57(e) (requiring warnings in labeling “as there is reasonable evidence of an association of a serious hazard with a drug; a causal relationship need not have been proved.”); 21 C.F.R. § 803.3 (adverse event reports address events possibly related to the drug or the device); 21 C.F.R. § 803.16 (adverse event report is not an admission of causation).
Kadane’s analysis of the case goes further astray when he suggests that the facts were strong enough for the case to have survived summary judgment. Id. at 9. The Matrixx case was a decision on the adequacy of the pleadings, not of the adequacy of the facts proven. Elsewhere, Kadane acknowledges the difference between a challenge to the pleadings and the legal sufficiency of the facts, id. at 7 & n.8, but Kadane asserts, without explanation, that the difference is “technical” and does not matter.” Not true. The motion to dismiss is made upon receipt of the plaintiffs’ complaint, but the motion for summary judgment is typically made at the close of discovery, on the eve of trial. The allegations can be conclusory, and they need have only plausible support in other alleged facts to survive a motion to dismiss. The case, however, must have evidence of all material facts, as well as expert witness opinion that survives judicial scrutiny for scientific validity under Rule 702, to survive a motion for summary judgment, which comes much later in the natural course of any litigated case.
Kadane appears to try to support the conflation of dismissals on the pleadings and summary judgments by offering a definition of summary judgment that is not quite accurate, and potentially misleading: “The idea behind summary judgment is that, even if every fact alleged by the opposing party were found to be true, the case would still fail for legal reasons.” Id. at 2. The problem is that at the summary judgment stage, as opposed to the pleading stage, the party with the burden of proof cannot rest upon his allegations, but must come forward with facts, not allegations, to support every essential element of his case. A plaintiff in a personal injury action (not a securities fraud case), for example, may easily survive a motion to dismiss by alleging medical causal connection, but at the summary judgment stage, that plaintiff must serve a report of an appropriately qualified expert witness, who in turn has presented a supporting opinion, reliably ground in science, to survive both evidentiary challenges and a dispositive motion.
Kadane concludes that the Matrixx decision’s “fact-based consideration” is consistent with a “Bayesian decision-theoretic approach that models how to make rational decisions under uncertainty.” Id. at 9. I am 99.99999% certain that Justice Sotomayor would not have a clue about what Professor Kadane was saying. Although statistical significance may have played no role in the Court’s holding, and in Kadane’s Bayesian decision-theoretic approach, I am 100% certain that the irrelevance of statistical significance to the Court’s and Prof. Kadane’s approaches is purely coincidental.
*First posted on Monday, February 6th at: http://schachtmanlaw.com/interstitial-doubts-about-the-matrixx/
Schactman’s legal practice focuses on the defense of product liability suits, with an emphasis on the scientific and medico-legal issues. He teaches a course in statistics in the law, at the Columbia Law School, NYC.
Error Statistics Philosophy 
Thanks so much for this post. I found it very interesting in first hearing of this case to learn that companies are not required to inform shareholders of news, simply because it is likely to alter a stock price, even if it is fairly obviously relevant to an investor’s reasonable analysis of the stock’s worth. It is only required to provide information which, if not revealed, would render misleading something the company already said, in this case, the fact that Matrixx had publicly denied reports claiming a link between its Zicom and loss of smell. I mean they were already battling lawsuits from users on this, they may even have settled some cases by then, I’m not sure. But clearly they had gotten letters from the FDA warning them to stop issuing these clean bills of health.
What baffles me is how this could have been taken on by the supreme court, given that there was no new law here, just the same ruling that had been in place. What do you think?
Of course, as you point out, the allegation that it indicates anything about the evidential basis for causal inference is faulty, but why take on this case?
The Matrixx case is a fascinating example of bad appellate strategy. As you know I have blogged a bit about this case, and the related personal injury litigation. The company was, at first, very successful in obtaining dismissals of the personal injury cases on grounds that the plaintiffs’ medical causation experts’ opinions were not “grounded in science.” The personal injury cases were then aggregated in federal court for pretrial handling, something known as an MDL – a multidistrict litigation. The defense made its so-called Daubert motion in the MDL, but lost this time. What had changed? A different judge, but also by then the FDA had called for withdrawal of the over the counter medication, Zicam. The company settled cases shortly after losing the Daubert motion in the MDL. Zicam inventor was later arrested and eventually pleaded guilty on charges related to importing and distributing untested, illegal bird flu vaccine. See http://www.huffingtonpost.com/2011/08/08/charles-hensleys-bird-flu-cure_n_921723.html
Your comments are exactly right: the company also brought the securities fraud case on itself by overstating projected sales and by affirmatively announcing the lack of health issues when they had notice of problems (even though the problems hardly amounted to sufficient evidence of causation). There are, of course, other laws that require disclosure, such as the Sarbanes-Oxley Act, but Rule 10b5 of the SEC is a way of creating a class of investors who have all been economically injured, not by the medication, but by the misleading information give to the market.
The appelllate strategy mistake? Well, it was hubris to take the reversal up to the Supreme Court. The defense’s misidentification of “materiality” with “causation,” which Prof. Kadane repeated, doomed the strategy from the inception. It was a unanimous opinion, which is rare these days. The defense did not pick up a single vote. At least with the benefit of hindsight, we can say that confirms it was a bad decision to appeal.
So it was the fault of the defense? But again, I’m mystified as to why the Supreme Court would take it up, as it was no new law, not that I’m aware of how they make those decisions, but am very curious in this case especially.
Was it the fault of the defense? The earlier decisions of the intermediate appellate courts certainly encouraged and inspired the company’s lawyers to make the motion to dismiss. I suspect that those lawyers viewed the reversal of the dismissal by the Ninth Circuit as a sign that they should appeal. The Ninth Circuit is the most reversed of the circuits in the US Supreme Court. I would add that the Supreme Court has narrowed the availability of class actions in several contexts (as in Wal-Mart v. Dukes), and it has been particularly limiting in the context of security fraud class actions. Obviously, all nine of the justices of the supposedly business-friendly Roberts Court thought that this case should go forward
Jay Kadane forwarded the following comment through e-mail:
Contrary to Mr. Schachtman’s comment, the purpose of my article is not to assess the importance of the Supreme Court’s decision in Matrixx v. Siracusano, but rather to understand how the courts appear to be construing “statistical significance’. From this perspective, both a rule 12b6 motion and a summary judgment motion require that all facts and reasonable inferences therefrom be taken as alleged by the non-moving party, here the plaintiff. In both motions, the question is whether, given those “facts”, the plaintiff’s case fails as a matter of law. While I appreciate that the distinction between a rule 12b6 motion ( before discovery is taken) and summary judgment (after discovery) is important to litigators, it is not a distinction critical to my inquiry. However Mr. Schachtman has a valid point that in my next-to-last paragraph, I should have written “a rule 12b6 motion” instead of ” summary judgement” when referring to the Matrixx decision.
I made no statement with respect to the extent to which Justice Sotomayor may appreciate the distinction between Bayesian and classical statistics. What I wrote is that the decisions are consistent with a Bayesian approach. But there are only three cases that provide evidence: fen-phen, Avandia and Zicam. No sensible statistician, Bayesian or classical, would draw strong conclusions on the basis of a sample of size three.
I realize that Prof. Kadane did not comment upon Justice Sotomayor’s appreciation of Bayesian statistics. That was simply my snarky comment. Putting that aside, I was determined to make a few things clear about what was decided, and what was simply extraneous musings we lawyers call obiter dicta. It is here that I fear Prof. Kadane’s misunderstands the relation of the Siracusano opinion in the Supreme Court to the lower courts’ opinions. I also fear I didn’t make myself clear, so I will try again.
1. Motions to dismiss on the pleadings are indeed based upon Federal Rule of Civil Procedure 12(b)(6), which requires courts to accept plausible allegations as true. The situation is substantially different for motions for summary judgment (Rule 56), which require the party opposing judgment to show that it can offer evidence on every material element of its claim or defense, sufficient to warrant the attention of a jury (or the judge if the trial court is sitting as the trier of fact). Just to pick one important example, it is relatively easy to allege that a product caused harm, and survive a 12b6 motion. It is quite another thing to show that you are prepared to offer testimonial or other evidence of causation to warrant a case’s submission to a jury. The required evidence of causation will usually be in the form of a proffered opinion of an expert witness, and that opinion must satisfy the requirements of Federal Rules of Evidence 702 and 703; that is, the opinion must satisfy what has come to be known as “Daubert,” and it must be based upon studies that are reasonable for an expert in the field to rely upon. So for summary judgments, allegations do not count any more, contrary to Prof. Kadane’s suggestion. This was more than a mere lapsus calami.
2. Prof. Kadane did not respond to my point that the Supreme Court did not hold that statistical significance was not required for proving causation. Neither causation nor withdrawal of an FDA license was at issue in Siracusano. The defense counsel improvidently and incorrectly injected causation into their argument in the hope of borrowing from various Daubert decisions and their insistence that chance (as well as bias and confounding) be ruled out before accepting an association as causal. The problem is that Daubert does not apply to pleadings (and when it came time to challenge the plaintiffs’ expert witnesses’ causal opinions in the personal injury cases, the company lost its motion before the same federal multi-district litigation judge who granted the motion to dismiss the securities class action). Defense counsel apparently thought that injecting causation into the securities fraud case would help them in their attempt to have the complaint dismissed. The defensive strategy proved clever by halves, to use a quantitative metaphor.
As the Supreme Court’s HOLDING teaches us, causation between the medication and the alleged harm is not required for a securities fraud case. (Causation between the deliberate falsehood and the economic loss is, however, required for the fraud action.) The irrelevance of medical causation to the securities fraud case implies that the failure to allege it, or the failure to allege statistical significance, or the total absence of statistical significance, is irrelevant to the holding of the case, which turns on whether plaintiffs alleged sufficient facts to show that information deliberately withheld or distorted would make a difference to regulatory action, which in turn would affect the company’s cheery predictions of market success in the future. As Prof. Kadane rightly observes, there was no opportunity to assess attained significance in the occurrence of anecdotes of anosmia among Zicam users. To make this assessment, we would have to know whether there was a baseline rate among persons with colds, and who did not use Zicam, or who used other cold medications. Imputing a denominator is typically pure speculation. See Harry Spiera, “Scleroderma After Silicone Augmentation Mammoplasty,” 260 J.
Am. Med. Ass’n 236 (1988) (claiming to discern an association between silicone breast implants and scleroderma in this speculative fashion). Similarly, disproportionality analyses are fraught with problems, but they are sometimes used to detect a signal of harm from the use of a particular medication.
3. To the extent that the Supreme Court held that medical causation was not required to make out a case of securities fraud, it disapproved of, if not outrightly overruled, the earlier Court of Appeals decisions. Each of those cases had a complex array of facts, involving close involvement of the FDA in the process of monitoring and assessing potential adverse outcomes. The legal significance of the FDA’s involvement in, and knowledge of, the facts supposedly not disclosed to the market is that the failure to disclose those facts to the market was not likely to result in an untoward surprise in the form of adverse regulatory action on the medication in question.
The main reason I wrote about Prof. Kadane’s article is that I remain concerned that he has written an article that confuses dicta with holding, allegations with facts, and medical causation with the basis for regulatory action. Nothing in his reply addresses the confusions I outline above.
NAS
I am starting a new internet website directory and was wondering if I can submit your website? I’m trying to grow my directory gradually by hand so that it retains top quality. I’ll make sure and put your blog in the correct category and I’ll additionally use, “” as your anchor text. Make sure you let me know if this is alright with you by mailing me at:. Bless you
tiffanyandcojewelrys.net http://www.tiffanyandcojewelrys.net
Tiffany, eh? I love their Atlas series with the roman numerals etched in gold. I have 2 necklaces and a bracelet from the Atlas series, one with rubies.