Hostname: page-component-78c5997874-s2hrs Total loading time: 0 Render date: 2024-11-04T18:33:12.415Z Has data issue: false hasContentIssue false
  • English
  • Français

A monotonicity in reversible Markov chains

Part of: Markov processes Special processes

Published online by Cambridge University Press:  14 July 2016

Robert Lund ,
Ying Zhao  and
Peter C. Kiessler
Robert Lund*
Affiliation:
Clemson University
Ying Zhao*
Affiliation:
The University of Georgia
Peter C. Kiessler*
Affiliation:
Clemson University
*
∗Postal address: Department of Mathematical Sciences, Clemson University, O-106 Martin Hall Box 340975, Clemson, SC 29634-0975, USA.
∗∗∗Postal address: Department of Statistics, The University of Georgia, Athens, GA 30602-1952, USA.
∗Postal address: Department of Mathematical Sciences, Clemson University, O-106 Martin Hall Box 340975, Clemson, SC 29634-0975, USA.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this paper we identify a monotonicity in all countable-state-space reversible Markov chains and examine several consequences of this structure. In particular, we show that the return times to every state in a reversible chain have a decreasing hazard rate on the subsequence of even times. This monotonicity is used to develop geometric convergence rate bounds for time-reversible Markov chains. Results relating the radius of convergence of the probability generating function of first return times to the chain's rate of convergence are presented. An effort is made to keep the exposition rudimentary.

Keywords

Convergence ratedecreasing hazard rateMarkov chainmonotonicityrenewal sequence

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 60K05: Renewal theory
Type
Research Papers
Information
Journal of Applied Probability , Volume 43 , Issue 2 , June 2006 , pp. 486 - 499
Copyright
© Applied Probability Trust 2006 

References

Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Prob. 15, 700–738.Google Scholar
Berenhaut, K. S. and Lund, R. (2001). Geometric renewal convergence rates from hazard rates. J. Appl. Prob. 38, 180–194.Google Scholar
Berenhaut, K. S. and Lund, R. (2002). Renewal convergence rates for DHR and NWU lifetimes. Prob. Eng. Inf. Sci. 16, 67–84.Google Scholar
Brown, M. (1980). Bounds, inequalities, and monotonicity properties for some specialized renewal processes. Ann. Prob. 8, 227–240.Google Scholar
Chen, M.-F. (2000). Equivalence of exponential ergodicity and L2-exponential convergence of Markov chains. Stoch. Process. Appl. 87, 281–297.Google Scholar
Chen, M.-F. (2005). Eigenvalues, Inequalities and Ergodic Theory. Springer, London.Google Scholar
Diaconis, P. and Stroock, D. (1991). Geometric bounds for eigenvalues of Markov chains. Ann. Appl. Prob. 1, 36–61.Google Scholar
Feller, W. (1968). An Introduction to Probability Theory and Its Applications, 3rd edn. John Wiley, New York.Google Scholar
Hansen, B. G. and Frenk, J. B. G. (1991). Some monotonicity properties of the delayed renewal function. J. Appl. Prob. 28, 811–821.Google Scholar
Kalashnikov, V. V. (1994). Topics on Regenerative Processes. CRC Press, Boca Raton, FL.CrossRefGoogle Scholar
Keilson, J. and Kester, A. (1978). Unimodality preservation in Markov chains. Stoch. Process. Appl. 7, 179–190.CrossRefGoogle Scholar
Kendall, D. G. (1959). Unitary dilations of Markov transition operators, and the corresponding integral representations for transition-probability matrices. In Probability and Statistics, ed. Grenander, U., John Wiley, New York, pp. 139–161.Google Scholar
Kijima, M. (1997). Markov Processes for Stochastic Modeling. Chapman and Hall, London.CrossRefGoogle Scholar
Kulkarni, V. G. (1995). Modeling and Analysis of Stochastic Systems. Chapman and Hall, London.Google Scholar
Liggett, T. M. (1989). Total positivity and renewal theory. In Probability, Statistics, and Mathematics, eds Anderson, T. W., Athreya, K. B. and Iglehart, D. L., Academic Press, Boston, MA, pp. 141–162.Google Scholar
Lindvall, E. T. (1992). Lectures on the Coupling Method. John Wiley, New York.Google Scholar
Lund, R. B. and Tweedie, R. L. (1996). Geometric convergence rates for stochastically ordered Markov chains. Math. Operat. Res. 21, 182–194.Google Scholar
Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, Berlin.CrossRefGoogle Scholar
Rosenthal, J. S. (1995). Convergence rates for Markov chains. SIAM Rev. 37, 387–405.Google Scholar
Ross, S. M. (1996). Stochastic Processes, 2nd edn. John Wiley, New York.Google Scholar
Sengupta, D., Chatterjee, A. and Chakraborty, B. (1995). Reliability bounds and other inequalities for discrete life distributions. Microelectron. Reliab. 35, 1473–1478.Google Scholar
Shaked, M. and Shanthikumar, J. G. (1987). IFRA properties of some Markov Jump processes with general state space. Math. Operat. Res. 12, 562–568.Google Scholar
Shaked, M. and Shanthikumar, J. G. (1994). Stochastic Orders and Their Applications. Academic Press, Boston, MA.Google Scholar
Stoyan, D. (1983). Comparison Methods for Queues and other Stochastic Models. John Wiley, Chichester.Google Scholar
Stroock, D. W. (2005). An Introduction to Markov Processes. Springer, Berlin.Google Scholar
Tuominen, P. and Tweedie, R. L. (1979). Exponential ergodicity in Markovian queueing and dam models. J. Appl. Prob. 16, 867–880.Google Scholar