3 - Entropy

Summary

In this chapter we extend the notion of entropy per lattice site from shift-invariant measures on finite lattices (see Example 1.2.1) to shift-invariant measures on infinite lattices and, more generally, to measures which are invariant under some (semi)group T of transformations. The presentation of the material is influenced by many textbooks, among them [38], [42], [43], [46], [60].

We start with a section where the notion of entropy of a probability vector from Section 1.1 is generalized to a concept of (conditional) entropy of partitions on general probability spaces. Dynamics are added to this in Section 3.2, where the entropy of a m.p.d.s. (X, B, T, μ) is defined and its basic properties are explored. In the last section we slightly change our point of view: since a given action T on a measurable space (X, B) may have many invariant measures, we look at the entropy of a dynamical system as a function of the measure.

Information and entropy of partitions

Let (X, B, μ) be a probability space. A collection α = {A_i : i ∈ I} of measurable subsets of X is a μ-partition of X if μ(A_i ∩ A_j) = 0 for i ≠ j, μ{X\∪_i∈IA_i) = 0 and μ(A_i) > 0 for all i ∈ I. Because of the last requirement, α is at most countable.