Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-25T08:07:23.921Z Has data issue: false hasContentIssue false
  • English
  • Français

An Efficient Procedure for Computing Quasi-Stationary Distributions of Markov Chains by Sparse Transition Structure

Part of: Markov processes Parametric inference

Published online by Cambridge University Press:  01 July 2016

P. K. Pollett  and
D. E. Stewart
P. K. Pollett*
Affiliation:
The University of Queensland
D. E. Stewart*
Affiliation:
The Australian National University
*
* Postal address: Department of Mathematics, The University of Queensland, QLD 4072, Australia.
** Postal address: Programme in Advanced Computation, School of Mathematical Sciences, The Australian National University, Canberra, ACT 2601, Australia.
Get access Rights & Permissions [Opens in a new window]

Abstract

We describe a computational procedure for evaluating the quasi-stationary distributions of a continuous-time Markov chain. Our method, which is an ‘iterative version' of Arnoldi's algorithm, is appropriate for dealing with cases where the matrix of transition rates is large and sparse, but does not exhibit a banded structure which might otherwise be usefully exploited. We illustrate the method with reference to an epidemic model and we compare the computed quasi-stationary distribution with an appropriate diffusion approximation.

Keywords

ARNOLDI ALGORITHMCONTINUOUS-TIME MARKOV CHAIN

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 62F15: Bayesian inference
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 26 , Issue 1 , March 1994 , pp. 68 - 79
Copyright
Copyright © Applied Probability Trust 1994 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This work was carried out with the support of an Australian Research Council grant and a University of Queensland Special Project Grant.

References

[1] Bartlett, M. S. (1956) Deterministic and stochastic models for recurrent epidemics. Proc. 3rd Berkeley Symp. Math. Statist. Prob. 4, 81–109.Google Scholar
[2] Bartlett, M. S. (1960) Some stochastic models in ecology and epidemiology. In Contributions to Probability and Statistics, A Volume dedicated to Harold Hotelling, pp. 89–96. Stanford University Press.Google Scholar
[3] Cavender, J. A. (1978) Quasistationary distributions for birth-death processes. Adv. Appl. Prob. 10, 570–586.Google Scholar
[4] Cullum, J. and Willoughby, R. (1978) Lanzcos and the computation in specified intervals of the spectrum of large, sparse real symmetric matrices. Sparse Matrix Proceedings, pp. 220–255.Google Scholar
[5] Dambrine, S. and Moreau, M. (1981) Note on the stochastic theory of a self-catalytic chemical reaction, I. Physica 106A, 559–573.CrossRefGoogle Scholar
[6] Dambrine, S. and Moreau, M. (1981) Note on the stochastic theory of a self-catalytic chemical reaction, II. Physica 106A, 574–588.Google Scholar
[7] Darroch, J. N. and Seneta, E. (1965) On quasi-stationary distributions in absorbing discrete-time finite Markov chains. J. Appl. Prob. 2, 88–100.Google Scholar
[8] Darroch, J. N. and Seneta, E. (1967) On quasi-stationary distributions in absorbing continuous-time finite Markov chains. J. Appl. Prob. 4, 192–196.Google Scholar
[9] Flaspohler, D. C. (1974) Quasi-stationary distributions for absorbing continuous-time denumerable Markov chains. Ann. Inst. Statist. Math. 26, 351–356.Google Scholar
[10] Golub, G. H. and Van Loan, C. (1989) Matrix Computations, 2nd edn. John Hopkins Press, Baltimore, MD.Google Scholar
[11] Good, P. (1968) The limiting behaviour of transient birth-death processes conditioned on survival. J. Austral. Math. Soc. 8, 716–722.Google Scholar
[12] Holling, C. S. (1973) Resilience and stability of ecological systems. Ann. Rev. Ecol. Systematics 4, 1–23.CrossRefGoogle Scholar
[13] Kendall, D. G. (1956) Deterministic and stochastic epidemics in closed populations. Proc. 3rd Berkeley Symp. Math. Statist. Prob. 4, 149–165.Google Scholar
[14] Kingman, J. F. C. (1963) The exponential decay of Markov transition probabilities. Proc. London Math. Soc. 13, 337–358.CrossRefGoogle Scholar
[15] Klein, D. R. (1968) The introduction, increase, and crash of reindeer on St. Matthew Island. J. Wildlife Man. 32, 351–367.Google Scholar
[16] Mandl, P. (1960) On the asymptotic behaviour of probabilities within groups of states of a homogeneous Markov process. Cas. Pest. Mat. 85, 448–456.Google Scholar
[17] Mech, L. D. (1966) The wolves of Isle Royale. Fauna of the National Parks: U.S. Fauna Series 7, U.S. Government Printing Office, Washington, DC.Google Scholar
[18] Nummelin, E. (1976) Limit theorems for a-recurrent semi-Markov processes. Adv. Appl. Prob. 8, 531–547.Google Scholar
[19] Oppenheim, I., Schuler, K. K. and Weiss, G. H. (1977) Stochastic theory of nonlinear rate processes with multiple stationary states. Physica 88A, 191–214.Google Scholar
[20] Parsons, R. W. and Pollett, P. K. (1987) Quasistationary distributions for some autocatalytic reactions. J. Statist. Phys. 46, 249–254.Google Scholar
[21] Pollett, P. K. (1988) On the problem of evaluating quasistationary distributions for open reaction schemes. J. Statist. Phys. 53, 1207–1215.Google Scholar
[22] Pollett, P. K. (1990) On a model for interference between searching insect parasites. J. Austral. Math. Soc. B 31, 133–150.Google Scholar
[23] Pollett, P. K. and Roberts, A. J. (1990) A description of the long-term behaviour of absorbing continuous-time Markov chains using a centre manifold. Adv. Appl. Prob. 22, 111–128.Google Scholar
[24] Ridler-Rowe, C. J. (1967) On a stochastic model for an epidemic. J. Appl. Prob. 4, 19–33.Google Scholar
[25] Saad, Y. (1980) Variations on Arnoldi's method for computing the eigenelements of large unsymmetric matrices. Linear Algebra Appl. 34, 269–295.Google Scholar
[26] Scheffer, V. B. (1951) The rise and fall of a reindeer herd. Sci. Monthly 73, 356–362.Google Scholar
[27] Seneta, E. (1973) Non-negative Matrices and Markov Chains. Springer-Verlag, New York.Google Scholar
[28] Seneta, E. and Vere-Jones, D. (1966) On quasi-stationary distributions in discrete-time Markov chains with a denumerable infinity of states. J. Appl. Prob. 3, 403–434.CrossRefGoogle Scholar
[29] Turner, J. W. and Malek-Mansour, M. (1978) On the absorbing zero boundary problem in birth and death processes. Physica 93A, 517–525.CrossRefGoogle Scholar
[30] Tweedie, R. L. (1973) The calculation of limit probabilites for denumerable Markov processes from infinitesimal properties. J. Appl. Prob. 10, 84–99.CrossRefGoogle Scholar
[31] Tweedie, R. L. (1974) Quasi-stationary distributions for Markov chains on a general state-space. J. Appl. Prob. 11, 726–741.CrossRefGoogle Scholar
[32] Vere-Jones, D. (1962) Geometric ergodicity in denumerable Markov chains. Quart. J. Math. (2) 13, 7–28.Google Scholar
[33] Vere-Jones, D. (1969) Some limit theorems for evanescent processes. Austral. J. Statist. 11, 67–78.Google Scholar
[34] Yaglom, A. M. (1947) Certain limit theorems of the theory of branching processes. Dokl. Acad. Nauk SSSR 56, 795–798.Google Scholar