Hostname: page-component-586b7cd67f-rcrh6 Total loading time: 0 Render date: 2024-11-28T20:44:58.678Z Has data issue: false hasContentIssue false
  • English
  • Français

Computing approximate stationary distributions for discrete Markov processes with banded infinitesimal generators

Published online by Cambridge University Press:  14 July 2016

Carlos F. Borges  and
Craig S. Peters
Carlos F. Borges*
Affiliation:
Naval Postgraduate School, Monterey
Craig S. Peters*
Affiliation:
General Electric Co., New York
*
∗Postal address: Code MA/BC, Naval Postgraduate School, Monterey, CA 93943, USA. Email address: [email protected]
∗∗Postal address: General Electric Co., Building K1, SC1D, One Research Circle, Niskayina, NY 12309, USA.
Get access Rights & Permissions [Opens in a new window]

Abstract

We develop an algorithm for computing approximations to the stationary distribution of a discrete birth-and-death process, provided that the infinitesimal generator is a banded matrix. We begin by computing stationary distributions for processes whose infinitesimal generators are Hessenberg. Our derivation in this special case is different from the classical case but it leads to the same result. We then show how to extend these ideas to processes where the infinitesimal generator is banded (or half-banded) and to quasi-birth–death processes. Finally, we give an example of the application of this method to a nearly completely decomposable Markov chain to demonstrate the general applicability of the technique.

Keywords

Homogeneous complementsingular value decomposition
Type
Research Papers
Information
Journal of Applied Probability , Volume 36 , Issue 4 , December 1999 , pp. 1086 - 1100
Copyright
Copyright © Applied Probability Trust 1999 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bai, Z., and Demmel, J. W. (1993). Computing the generalized singular value decomposition. SIAM J. Scientific Computing 14, 1464–1486.Google Scholar
Courtois, P. J. (1977). Decomposability: Queueing and Computer System Applications. Academic Press, New York.Google Scholar
Koury, R., McAllister, D. F., and Stewart, W. J. (1984). Methods for computing the stationary distributions of nearly-completely-decomposable Markov chains. SIAM J. Algebraic Discrete Mathematics 5, 164–186.Google Scholar
Meyer, C. D. (1989). Stochastic complementation, uncoupling Markov chains and the theory of nearly reducible systems. SIAM Review 31, 240–272.Google Scholar
Simon, H. A., and Ando, A. (1961). Aggregation of variables in dynamic systems. Econometrica 29, 111–138.CrossRefGoogle Scholar
Stewart, W. J. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press.Google Scholar
Takahashi, Y. (1975). A Lumping Method for Numerical Calculation of Stationary Distributions of Markov Chains. Technical Report B-18, Department of Information Sciences, Tokyo Institute of Technology, Tokyo, Japan.Google Scholar
Van Loan, C. F. (1976). Generalizing the singular value decomposition. SIAM J. Numerical Analysis 13, 76–83.Google Scholar