2 - The probabilistic turn

Summary

Discussion of parsimony took a probabilistic turn in the twentieth century. The project was to use probability theory to analyze and justify Ockham's razor. Not all of these efforts succeeded, but two of them did. I think there are two “parsimony paradigms” in which probability ideas show that parsimony is epistemically relevant. The two paradigms were developed within two different philosophical frameworks for understanding probability; one paradigm finds its home in Bayesianism, the other in frequentism. To set the stage for investigating probabilistic approaches to Ockham's razor, I'll start this chapter by providing a brief (and I hope accessible) primer on probability. But first I want to say a little about Bayesianism and frequentism.

Two philosophies of probability

Bayesianism is a philosophy of inference that traces back to a mathematical result (a theorem) obtained by Thomas Bayes (1701–1761). Bayes's (1764) theorem describes how the probability you assign to a hypothesis should be influenced by the new evidence you acquire. Bayesianism is now a general philosophy of scientific reasoning that has grown richer and more detailed than its eighteenth-century beginnings. This philosophy says that scientific reasoning has the attainable goal of figuring out how probable different scientific hypotheses are, given the evidence at hand. Or more modestly, it maintains that science is in the business of figuring out which hypotheses are more probable than which others, again in the light of the evidence. Either way, science crucially involves thinking about the probabilities of hypotheses.

Bayesianism was not the dominant philosophy of probabilistic inference that scientists themselves embraced in the twentieth century. Rather, the dominant mode of thought was frequentism. Frequentism does not have the simple unity that Bayesianism exhibits; rather, it is a varied collection of ideas about how observations should be used to evaluate hypotheses. Frequentism uses probability ideas in this enterprise just as Bayesianism does, but its basic idea is different. The first commandment of frequentism is: thou shalt not talk about the probabilities that hypotheses have! The claim that science has the job of assessing how probable different theories are may sound like an unremarkable truism, but this innocent-sounding remark is something that frequentists categorically reject.

The difference between frequentism and Bayesianism is often characterized in terms of what each philosophy takes the concept of probability to mean.