Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-27T20:26:22.313Z Has data issue: false hasContentIssue false
  • English
  • Français

Infinite block-structured transition matrices and their properties

Part of: Markov processes Operations research and management science

Published online by Cambridge University Press:  01 July 2016

Yiqiang Q. Zhao ,
Wei Li  and
W. John Braun
Yiqiang Q. Zhao*
Affiliation:
University of Winnipeg
Wei Li*
Affiliation:
University of Winnipeg, Chinese Academy of Sciences
W. John Braun*
Affiliation:
University of Winnipeg
*
∗ Postal address: Department of Mathematics and Statistics, University of Winnipeg, Winnipeg, Canada R3B 2E9
∗ Postal address: Department of Mathematics and Statistics, University of Winnipeg, Winnipeg, Canada R3B 2E9
∗ Postal address: Department of Mathematics and Statistics, University of Winnipeg, Winnipeg, Canada R3B 2E9
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we study Markov chains with infinite state block-structured transition matrices, whose states are partitioned into levels according to the block structure, and various associated measures. Roughly speaking, these measures involve first passage times or expected numbers of visits to certain levels without hitting other levels. They are very important and often play a key role in the study of a Markov chain. Necessary and/or sufficient conditions are obtained for a Markov chain to be positive recurrent, recurrent, or transient in terms of these measures. Results are obtained for general irreducible Markov chains as well as those with transition matrices possessing some block structure. We also discuss the decomposition or the factorization of the characteristic equations of these measures. In the scalar case, we locate the zeros of these characteristic functions and therefore use these zeros to characterize a Markov chain. Examples and various remarks are given to illustrate some of the results.

Keywords

Infinite-state Markov chainsblock-form matricesMarkov chains with repeating rowstransient, recurrent and positive recurrent statescensored Markov chainsfactorization of generating functions

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 90B22: Queues and service 60J27: Continuous-time Markov processes on discrete state spaces
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 30 , Issue 2 , June 1998 , pp. 365 - 384
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Asmussen, S. (1987). Applied Probability and Queues. Wiley, Chichester.Google Scholar
[2] Freedman, D. (1983). Approximating Countable Markov Chains, 2nd edn. Springer, New York.Google Scholar
[3] Gibson, D. and Seneta, E. (1987). Augmented truncations of infinite stochastic matrices. J. Appl. Prob. 24, 600–608.Google Scholar
[4] Grassmann, W.K. (1985). The factorization of queueing equations and their interpretation. J. Opl. Res. Soc. 36, 1041–1050.Google Scholar
[5] Grassmann, W. K. and Heyman, D. P. (1990). Equilibrium distribution of block-structured Markov chains with repeating rows. J. Appl. Prob. 27, 557–576.CrossRefGoogle Scholar
[6] Grassmann, W. K. and Heyman, D. P. (1993). Computation of steady-state probabilities for infinite-state Markov chains with repeating rows. ORSA J. Computing 5, 292–303.CrossRefGoogle Scholar
[7] Haight, F. A. (1958). Two queues in parallel. Biometrika 45, 401–410.Google Scholar
[8] Heyman, D.P. (1995). Accurate computation of the fundamental matrix of a Markov chain. SIAM J. Matrix Anal. Appl. 16, 954–963.CrossRefGoogle Scholar
[9] Karlin, S. and Taylor, H. M. (1981). A Second Course in Stochastic Processes. Academic Press, New York.Google Scholar
[10] Kemeny, J. G., Snell, J. L. and Knapp, A. W. (1976). Denumerable Markov Chains, 2nd edn. Springer, New York.Google Scholar
[11] Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach. The Johns Hopkins University Press, Baltimore.Google Scholar
[12] Neuts, M. F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Decker Inc., New York.Google Scholar