5 - Probability Theory

Summary

Outcomes of an experiment

Consider an experiment that depends on chance, and that does not depend on any unknown information. What can we say about the outcome? If we actually carry the experiment out, we can say precisely what happened – but what can we say beforehand? The best description we can hope to give is one that specifies the likelihood of each possible outcome. Such a description is called a probability distribution. Probability theory is a way of reasoning about likelihoods.

Consider the roll of a die – there are six possible outcomes (not including wacky things like “the die rolls right off the table”). We implicitly assume that outcomes of a single experiment are mutually exclusive. That is, only one of the outcomes can occur each time that the experiment is performed.

The possible outcomes are shown in Figure 5.1.

The set of possible outcomes is called the sample space. It is also called the probability space.

Probabilities of outcomes

We describe the relative likelihoods of the six possible outcomes by assigning each outcome a number that represents its probability. If, for example, one outcome is twice as likely to occur as another, we assign the first outcome a probability twice that assigned to the second.

Certain conventions govern the numbers we use as probabilities. It would not make sense for one outcome to be −1 times as likely as another, so we restrict probabilities to be nonnegative numbers. We want probability 0 to correspond to impossibility; an outcome that never occurs would be assigned probability 0. We want probability 1 to correspond to certainty; an outcome that always occurs would be assigned probability 1. In a typical experiment, each outcome's probability is a number bigger than 0 and smaller than 1.