Book contents
- Frontmatter
- Contents
- Preface to the second edition
- Preface to the first edition
- 1 Introduction
- 2 Combinatorics
- 3 Sets and measures
- 4 Probability
- 5 Discrete random variables
- 6 Information and entropy
- 7 Communication
- 8 Random variables with probability density functions
- 9 Random vectors
- 10 Markov chains and their entropy
- Exploring further
- Appendix 1 Proof by mathematical induction
- Appendix 2 Lagrange multipliers
- Appendix 3 Integration of exp(−½x2)
- Appendix 4 Table of probabilities associated with the standard normal distribution
- Appendix 5 A rapid review of matrix algebra
- Selected solutions
- Index
10 - Markov chains and their entropy
Published online by Cambridge University Press: 06 July 2010
- Frontmatter
- Contents
- Preface to the second edition
- Preface to the first edition
- 1 Introduction
- 2 Combinatorics
- 3 Sets and measures
- 4 Probability
- 5 Discrete random variables
- 6 Information and entropy
- 7 Communication
- 8 Random variables with probability density functions
- 9 Random vectors
- 10 Markov chains and their entropy
- Exploring further
- Appendix 1 Proof by mathematical induction
- Appendix 2 Lagrange multipliers
- Appendix 3 Integration of exp(−½x2)
- Appendix 4 Table of probabilities associated with the standard normal distribution
- Appendix 5 A rapid review of matrix algebra
- Selected solutions
- Index
Summary
Stochastic processes
So far in this book we have tended to deal with one (or at most two) random variables at a time. In many concrete situations, we want to study the interaction of ‘chance’ with ‘time’, e.g. the behaviour of shares in a company on the stock market, the spread of an epidemic or the movement of a pollen grain in water (Brownian motion). To model this, we need a family of random variables (all defined on the same probability space), (X(t), t ≥ 0), where X(t) represents, for example, the value of the share at time t.
(X(t), t ≥ 0) is called a (continuous time) stochastic process or random process. The word ‘stochastic’ comes from the Greek for ‘pertaining to chance’. Quite often, we will just use the word ‘process’ for short.
For many studies, both theoretical and practical, we discretise time and replace the continuous interval [0, ∞) with the discrete set ℤ+ = ℕ ∪ {0} or sometimes ℕ. We then have a (discrete time) stochastic process (Xn, n ∈ ℤ+). We will focus entirely on the discrete time case in this chapter.
Note. Be aware that X(t) and Xt (and similarly X(n) and Xn) are both used interchangeably in the literature on this subject.
There is no general theory of stochastic processes worth developing at this level. It is usual to focus on certain classes of process which have interesting properties for either theoretical development, practical application, or both of these. We will study Markov chains in this chapter.
- Type
- Chapter
- Information
- Probability and InformationAn Integrated Approach, pp. 217 - 244Publisher: Cambridge University PressPrint publication year: 2008