Posts Tagged With: FDA

PhilStatLaw: “Let’s Require Health Claims to Be ‘Evidence Based'” (Schachtman)

I see that Nathan Schachtman has had many interesting posts during the time I was away.  His recent post endorses the idea of “a hierarchy of evidence”–but philosophers of “evidence-based” medicine generally question or oppose it, at least partly because of disagreement as to where to place RCTs in the hierarchy.  What do people think?

Litigation arising from the FDA’s refusal to approval “health claims” for foods and dietary supplements is a fertile area for disputes over the interpretation of statistical evidence.  A ‘‘health claim’’ is ‘‘any claim made on the label or in labeling of a food, including a dietary supplement, that expressly or by implication … characterizes the relationship of any substance to a disease or health-related condition.’’ 21 C.F.R. § 101.14(a)(1); see also 21 U.S.C. § 343(r)(1)(A)-(B).

Unlike the federal courts exercising their gatekeeping responsibility, the FDA has committed to pre-specified principles of interpretation and evaluation. By regulation, the FDA gives notice of standards for evaluating complex evidentiary displays for the ‘‘significant scientific agreement’’ required for approving a food or dietary supplement health claim.  21 C.F.R. § 101.14.  SeeFDA – Guidance for Industry: Evidence-Based Review System for the Scientific Evaluation of Health Claims – Final (2009).

If the FDA’s refusal to approve a health claim requires pre-specified criteria of evaluation, then we should be asking ourselves why have the federal courts failed to develop a set of criteria for evaluating health effects claims as part of its Rule 702 (“Daubert“) gatekeeping responsibilities.  Why, after close to 20 years after the Supreme Court decided Daubert, can lawyers make “health claims” without having to satisfy evidence-based criteria?

Read the rest.

Categories: philosophy of science, Statistics | Tags: , , ,

Distortions in the Court? (PhilStock Feb 8)

Anyone who trades in biotech stocks knows that the slightest piece of news, rumors of successful /unsuccessful drug trials, upcoming FDA panels, anecdotal side effects, and much, much else, can radically alter a stock price in the space of a few hours.  Pre-market, for example, websites are busy disseminating bits of information garnered from anywhere and everywhere, helping to pump or dump biotechs.  I think just about every small biotech stock I’ve ever traded has been involved in some kind of lawsuit regarding what the company should have told shareholders during earnings.  (Most don’t go very far.)  If you ever visit the FDA page, you can find every drug/medical device coming up for considerations, recent letters to the company etc., etc.

Nevertheless, you might be surprised to learn that companies are not required to inform shareholders of news simply because it is likely to be relevant to an investor’s overall cost-benefit analysis in deciding how much the stock is worth, and where its price is likely to move. It’s more minimalist than that.  It is only required to provide information which, if not revealed, would render misleading something the company already said.

So for example suppose a drug company M publicly denied any reports claiming a link between its drug Z and effect E, declaring that drug Z had a clean bill of health as regards this risk concern.  Having made that statement, the company would then be in violation of the requirements if they did not also reveal information such as: numerous consumers were suing them alleging the untoward effect E from having taken drug Z; several letters had been written to the company from the FDA expressing concern about the number of cases where doctors had reported effect E among patients taking drug Z, still other letters warning company M that they should cease and desist from issuing statements that any alleged links between drug Z and effect E were entirely baseless and unfounded.

Now the information that company M was not revealing did not, and could not, have shown a statistically significant correlation between drug Z and effect E.  But failing to reveal this information rendered company M in violation of FDA and stock rules, because of the statements company M already made about drug Z’s clean bill of health regarding this very effect E (along with bullish price projections).  Not revealing this information, and the related information in their possession, rendered misleading things the company already said when it comes to information shareholders use in deciding on M’s value.

Pretty obvious, right?

Suppose then that company M is found in violation of this rule.  And suppose someone inferred from this that evidence of statistical significance is not required for showing a causal connection between a drug and hazardous side-effects.

Well, to infer that would be like doubly (or perhaps triply) missing the point: the ruling had nothing to do with what’s required to show cause and effect, but only what information a company is required to reveal to its shareholders in order not to mislead them (as regards information that could be of relevance to them in their cost-benefit assessments of the stock’s value and future price).

Secondly, the ruling made it very explicit that it was not making any claim about the actual existence of evidence linking drug Z and effect E: they were only proclaiming that drug company M would be in error, if they claimed they did not violate the rule of disclosure.[i]  (Determining whether there is any link between Z and E was an entirely separate matter.)

This is precisely  the situation as regards a drug company Matrixx, over the counter cold remedy Zicam, and side effect E: anosmia (loss or diminished sense of smell).  It was the focus of lawyer and guest blogger Nathan Schachtman yesterday.

“The potentially fraudulent aspect of Matrixx’s conduct was not that it had “hidden” adverse event reports, but rather that it had adverse event reports and a good deal of additional information, none of which it had disclosed to investors, when at the same time, the company chose to give the investment community particularly bullish projections of future sales.” (Schachtman)

Nevertheless, critics of statistical significance testing wasted no time in declaring that this ruling (which for some inexplicable reason made it to the Supreme Court) just goes to show that statistical significance is not and should not be required to show evidence of a causal link[ii]. (See also my Sept. 26 post).  Kadane’s article, which is quite interesting, concludes:

“The fact-based consideration that the Supreme Court endorses is very much in line with the Bayesian decision-theoretic approach that models how to make rational decisions under uncertainty. The presence or absence of statistical significance (in the formal, narrow sense) plays little role in such an analysis. “ (Jay Kadane)

I leave it to interested readers to explore the various  ins and outs of the case, which our guest poster has summarized in a much more legally correct fashion.

[i] Company M would certainly be in error, if the reason they claimed  not to have violated the rule of disclosure is that the information they did not reveal could not have constituted evidence of a statistically significant link between drug Z and effect E!

[ii] There was a session at the ASA last summer on this, including Kadane, Ziliac, and I don’t know who else (I had to leave prior to it).

Categories: Philosophy of Statistics | Tags: , , ,

Blog at