Appendix A - Probability

Summary

Random variables

Probability is an essential idea in both information theory and quantum mechanics. It is a highly developed mathematical and philosophical subject of its own, worthy of serious study. In this brief appendix, however, we can only sketch a few elementary concepts, tools, and theorems that we use elsewhere in the text.

In discussing the properties of a collection of sets, it is often useful to suppose that they are all subsets of an overall “universe” set U. The universe serves as a frame within which unions, intersections, complements, and other set operations can be described. In much the same way, the ideas of probability exist with a frame called a probability space Σ. For simplicity, we will consider only the discrete case. Then Σ consists of an underlying set of points and an assignment of probabilities. The set is called a sample space and its elements are events. The probability function, or probability distribution, assigns to each event e a real number P(e) between 0 and 1, such that the sum of all the probabilities is 1.

The probability P(e) is a measure of the likelihood that event e occurs. An impossible event has P(e) = 0 and a certain event would have P(e) = 1; in other cases, P(e) has some intermediate value.

The probability space itself contains all possible events that may occur, identified in complete detail, which makes it too elaborate for actual use. Suppose we flip a coin.