.
Some time, around the 1980s, philosophers of science turned their attention to scientific experiments in a way that contrasted with the reigning approaches to philosophy of science. My colleague, Wendy Parker, and I decided to embark on an experiment of our own, aimed at elucidating some central themes of this evolving movement, sometimes referred to as the ‘new experimentalism.’ It was to begin tomorrow, but due to unexpected weather conditions, I’ll be traveling back then, and find myself with an additional afternoon in New York City. So I’ll take this opportunity to begin my reflections, with the expectation of later incorporating Wendy’s insights, and refining my own. The philosophical concepts and ideas stemming from the philosophy of experiment provide powerful tools for making progress on fundamental problems of how we find things out in the face of limitations of data, models, and methods. The time is ripe for a comprehensive examination of this field, but our “experiment on experiment” here will be the bare beginnings of themes that come to mind. Please suggest corrections and additions in the comments.
Experiments have lives of their own.
The “lives of their own” slogan has many offshoots that crop up in this post. Notably, instead of viewing the role of experiments as handmaidens for testing theories, there’s the recognition that experiments and instruments have the power to shape experimental knowledge and create new, unexpected, phenomena leading to new theories. Moreover, we don’t need true theories to learn from intervening successfully: we may rely on types of experimental arguments.
Arguments from coincidence and entity realism.
Philosophers of science often associated with the new experimentalism are Ian Hacking (1983) and Nancy Cartwright (1983). An idea of “experimental realism” or “entity realism” growing out of their work underscores a canonical type of argument that may be called an “argument from coincidence”. If a phenomenon consistently occurs across deliberately altered experimental setups, and especially if we can intervene to modify the effects in predictable ways, there is strong evidence that the observed effects are not due to mere background variability, artifacts of the apparatus, or experimental errors. We have evidence of a genuine or reliable effect that will not go away. We might infer we have gotten hold of a real entity, without claiming to have true theories about it. Or so say entity realists. (Do you agree?) Another variant is in terms of arguments for common causes.
An example Hacking and Cartwright use to illustrate an argument from coincidence is Jean Perrin’s inference about Brownian motion from experiments on 13 distinct phenomena.(A similar argument is in Salmon 1984.) An argument from coincidence is an argument for ruling out coincidence. An open question is whether the associated conception of “entity realism” blurs a distinction between inferring a real effect versus inferring a real entity. To avoid blurring what may be reliably inferred, we may distinguish a variety of “arguments from” error, for different errors understood broadly: random, causal, generalization errors and the like.
Theory-independence.
Experiment is importantly independent from theory, both in its goals and justification. As for theory-independent goals, scientists may just be interested in collecting experimental knowledge—knowledge of what can be expected were specific interventions carried out. They may spend years building up novel ways to model, control, and probe phenomena, get good at estimating effects and learning how to check if things can go wrong. As for justification, it’s not that experimental inference is theory-free but that its assumptions need not rely on the theories under test, or on uncertain hypotheses that would threaten the experimental argument. For example, a properly designed randomized control trial enables a p-value to be computed, without further assumptions.
Historical episodes, practice, and context.
The interest in history and philosophy of science developed along with the philosophy of experiment (at least to this philosopher of science). It began to be recognized that in order for philosophies of science to be relevant to practice, they need to be examined and tested in relation to historical episodes. There was also the novel idea, underscored by Popper and Lakatos, that data and hypotheses could not be analyzed without considering how they were generated and selected—their history. Discovery and justification may not be separate after all. If they are selected in a biased fashion, that alters the warrant for inference.
Beyond logics of evidence, confirmation, explanation.
Looking at the logical relationships between statements of theory and evidence “e” without opening the black box of the data generation, modeling, instruments, social context, etc. leads to vacuous accounts.
Models of inquiry.
Scientific inquiry doesn’t operate with data e and hypotheses Hi, but with models of data, models of experiment, and models of substantive questions, theories, instruments, etc.
Relevance to Practice.
Examining experimental episodes is the key to developing a philosophy of science relevant for solving foundational, conceptual, and methodological problems in scientific practice. Our “outsider” status can help, rather than hinder, the critical appraisal of controversies about evidence policy. This is a great big open problem area.
Large-scale theory vs localism.
Interestingly, the philosophy of experiment grew up in a philosophy of science that was shifting from the (logical empiricist) accounts of confirmation and logics of induction to more holist entities such as Kuhnian paradigms with their mix of theories and sociology (ontology, methodology, aims, goals, and values). At the same time, attention to experiment emphasizes local contexts and piecemeal analysis. Actual inquiries, the new experimentalists show, focus on manifold local tasks: checking instruments, ruling out extraneous factors, getting accuracy estimates, distinguishing real effect from artifact, and estimating the effects of background factors (Kuhnian “normal” science). This large-scale vs piecemeal tension is another open issue in philosophy of science.
Duhemian problems of pinpointing blame for anomalies.
Scientists look to local experiments and strategies, including randomized control trials, simulation, and cross-validation, in order to provide keys to disentangle the sources of observed effects. Experiment is always local. Which alternative theories need to be ruled out for a particular inference to be reliable? In my case, studying the nitty-gritty details of the data collection and analysis from experimental episodes proved to be a gold mine.
…….
These themes are pretty much off the top of my head: it was an experiment during my last afternoon in NYC.[1] They are not well-organized or complete. I look forward to hearing from Wendy Parker, and from readers of this blog.
[1] Some are from Mayo 1996: Error and the Growth of Experimental Knowledge (Chicago 1996).
RERERENCES
Cartwright, N. 1983. How the laws of physics lie. Oxford: Clarendon Press.
Hacking, I. 1983. Representing and intervening: Introductory topics in the philosophy of natural science. Cambridge: Cambridge University Press.
Error Statistics Philosophy 
Just off the top of my head (as Data says) this evokes memories of the time I embarked on my own “Inquiry Into Inquiry”. Here’s a link.
• Inquiry Driven Systems • Inquiry Into Inquiry
Fantastic post! Thanks for sharing…
I would love to hear more about the history of the field!! I am most curious about your own experience of coming up with the ideas, and I am somewhat curious about the influence of Patrick Suppes to the field.
Also, in Chalmer’s book, “What Is This Thing Called Science?” it was described like Bayesiansm was influential in the philosophy of science before New Experimentalism emerged. I wonder how the trend of Bayesianism in the philosophy of science appeared to you. Was skepticism and criticism on realism, like Van Fraassen’s (if I am understanding correctly), popular? It would be interesting to know the historical context of New Experimentalism in the philosophy of science.
In many domains, experiments are designed and conducted to gain knowledge, not necessarily to test theories. Statistically designed experiments are both efficient and effective. See for a modern treatment see https://link.springer.com/book/10.1007/978-3-031-28482-3 , for a classical text see https://www.wiley.com/en-gb/Planning+of+Experiments-p-9780471574293
Also, “disentangling the sources of observed effects” is a worthy effort because it permits to generalize findings.
Reproducibility is another feature of experiments. Some are conducted just for this reason.
So, 1) gain knowledge, 2) generalize findings and 3) prove reproducibility are three important objectives of experiments, way beyond testing theories.
In terms of the “new experimentalism”, perhaps this should dwell on the combination of empirical experiments with simulations and first principles knowledge (physics, chemistry etc..). For a starter on computer experiments see https://onlinelibrary.wiley.com/doi/abs/10.1002/adts.202000254
Yes, those points are very much in the spirit of the philosophy of experiment. I do mean to add simulation and generalization. However, in my view, in using experiments to find things out, even though there isn’t a prespecified hypothesis, there is a question or problem, and before claiming to have decent evidence for an answer to that question, one wants to show that certain ways it could be wrong have been probed and fairly well ruled out. So it’s the same experimental reasoning. Thank you.
Mayo – I think one should distinguish the industrial ecosystem from the scientific one. Learning well and fast is a primary objective in industry were one can tolerate dead ends that lead to restarts. The standards of scientific papers is not necessarily of essence. For a glimpse at all this see https://link.springer.com/article/10.1007/s10845-021-01817-9. See also https://onlinelibrary.wiley.com/doi/abs/10.1002/qre.3449