Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-08T08:34:02.098Z Has data issue: false hasContentIssue false
  • English
  • Français

On the relationship between µ-invariant measures and quasi-stationary distributions for continuous-time Markov chains

Part of: Markov processes

Published online by Cambridge University Press:  01 July 2016

M. G. Nair  and
P. K. Pollett
M. G. Nair
Affiliation:
University of Queensland
P. K. Pollett*
Affiliation:
University of Queensland
*
∗∗Postal address: Department of Mathematics, The University of Queensland, QLD 4072, Australia. E-mail address: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In a recent paper, van Doorn (1991) explained how quasi-stationary distributions for an absorbing birth-death process could be determined from the transition rates of the process, thus generalizing earlier work of Cavender (1978). In this paper we shall show that many of van Doorn's results can be extended to deal with an arbitrary continuous-time Markov chain over a countable state space, consisting of an irreducible class, C, and an absorbing state, 0, which is accessible from C. Some of our results are extensions of theorems proved for honest chains in Pollett and Vere-Jones (1992).

In Section 3 we prove that a probability distribution on C is a quasi-stationary distribution if and only if it is a µ-invariant measure for the transition function, P. We shall also show that if m is a quasi-stationary distribution for P, then a necessary and sufficient condition for m to be µ-invariant for Q is that P satisfies the Kolmogorov forward equations over C. When the remaining forward equations hold, the quasi-stationary distribution must satisfy a set of ‘residual equations' involving the transition rates into the absorbing state. The residual equations allow us to determine the value of µ for which the quasi-stationary distribution is µ-invariant for P. We also prove some more general results giving bounds on the values of µ for which a convergent measure can be a µ-subinvariant and then µ-invariant measure for P. The remainder of the paper is devoted to the question of when a convergent µ-subinvariant measure, m, for Q is a quasi-stationary distribution. Section 4 establishes a necessary and sufficient condition for m to be a quasi-stationary distribution for the minimal chain. In Section 5 we consider ‘single-exit' chains. We derive a necessary and sufficient condition for there to exist a process for which m is a quasi-stationary distribution. Under this condition all such processes can be specified explicitly through their resolvents. The results proved here allow us to conclude that the bounds for µ obtained in Section 3 are, in fact, tight. Finally, in Section 6, we illustrate our results by way of two examples: regular birth-death processes and a pure-birth process with absorption.

Keywords

Q-PROCESSESSTOCHASTIC MODELSBIRTH-DEATH PROCESSES

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 60J35: Transition functions, generators and resolvents
Type
Research Article
Information
Advances in Applied Probability , Volume 25 , Issue 1 , March 1993 , pp. 82 - 102
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

∗

Present address: School of Mathematics and Statistics, Curtin University of Technology, Perth, WA6001, Australia. E-mail address: [email protected]

This research was funded under an Australian Research Council Grant and a University of Queensland Special Project Grant.

References

Cavender, J. A. (1978) Quasistationary distributions for birth-death processes. Adv. Appl. Prob. 10, 570–586.CrossRefGoogle Scholar
Chung, K. L. (1967) Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer-Verlag, Berlin.Google Scholar
Van Doorn, E. A. (1991) Quasi-stationary distributions and convergence to quasi-stationarity of birth–death processes. Adv. Appl. Prob. 23, 683–700.Google Scholar
Kelly, F. P. and Pollett, P. K. (1983) Sojourn times in closed queueing networks. Adv. Appl. Prob. 15, 638–656.Google Scholar
Kingman, J. F. C. (1963) The exponential decay of Markov transition probabilities. Proc. London Math. Soc. 13, 337–358.Google Scholar
Knopp, K. (1956) Infinite Sequences and Series. Dover, New York.Google Scholar
Lamb, C. W. (1971) On the construction of certain transition functions. Ann. Math. Statist. 42, 439–450.Google Scholar
Pollett, P. K. (1986) On the equivalence of µ-invariant measures for the minimal process and its q-matrix. Stoch. Proc. Appl. 22, 203–221.Google Scholar
Pollett, P. K. (1988) Reversibility, invariance and µ-invariance. Adv. Appl. Prob. 20, 600–621.CrossRefGoogle Scholar
Pollett, P. K. (1991a) On the construction problem for single-exit Markov chains. Bull. Austral. Math. Soc. 43, 439–450.Google Scholar
Pollett, P. K. (1991b) Invariant measures for Q-processes when Q is not regular. Adv. Appl. Prob. 23, 277–292.Google Scholar
Pollett, P. K. and Vere-Jones, D. (1992) A note on evanescent processes. Austral. J. Statist. To appear.Google Scholar
Reuter, G. E. H. (1957) Denumerable Markov processes and the associated contraction semi-groups on l. Acta Math. 97, 1–46.Google Scholar
Reuter, G. E. H. (1959) Denumerable Markov processes (II). J. London Math. Soc. 34, 81–91.Google Scholar
Reuter, G. E. H. (1962) Denumerable Markov processes (III). J. London Math. Soc. 37, 63–73.CrossRefGoogle Scholar
Tweedie, R. L. (1974) Some ergodic properties of the Feller minimal process. Quart. J. Math. Oxford 25, 485–495.Google Scholar
Vere-Jones, D. (1967) Ergodic properties on non-negative matrices I. Pacific J. Math. 22, 361–386.CrossRefGoogle Scholar
Vere-Jones, D. (1969) Some limit theorems for evanescent processes. Austral. J. Statist. 11, 67–78.CrossRefGoogle Scholar