Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-12-02T19:36:17.279Z Has data issue: false hasContentIssue false
  • English
  • Français

COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION

Published online by Cambridge University Press:  09 July 2015

F.M. Spieksma
F.M. Spieksma*
Affiliation:
Mathematics Institute, Leiden University, Niels Borhweg 1, 2333CA Leiden, The Netherlands Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The existence of a moment function satisfying a drift function condition is well known to guarantee non-explosiveness of the associated minimal Markov process (cf. [1,9]), under standard technical conditions. Surprisingly, the reverse is true as well for a countable space Markov process. We prove this result by showing that recurrence of an associated jump process, that we call the α-jump process, is equivalent to non-explosiveness. Non-explosiveness corresponds in a natural way to the validity of the Kolmogorov integral relation for the function identically equal to 1. In particular, we show that positive recurrence of the α-jump chain implies that all bounded functions satisfy the Kolmogorov integral relation. We present a drift function criterion characterizing positive recurrence of this α-jump chain.

Suppose that to a drift function V there corresponds another drift function W, which is a moment with respect to V. Via a transformation argument, the above relations hold for the transformed process with respect to V. Transferring the results back to the original process, allows to characterize the V-bounded functions that satisfy the Kolmogorov forward equation.

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 29 , Issue 4 , October 2015 , pp. 623 - 637
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Anderson, W.J. (1991). Continuous-time Markov chains. New York: Springer-Verlag.Google Scholar
2. Avrachenkov, K., Piunovskiy, A., and Zhang, Y. (2013). Markov processes with restart. J. Appl. Prob. 50: 960–968.Google Scholar
3. Brémaud, P. (1999). Markov chains. Gibbs fields, Monte Carlo Simulation, and Queues. Number 31 in Texts in Applied Mathematics. New York: Springer-Verlag.Google Scholar
4. Chen, M.F. (1986). Coupling for jump processes. Acta Mathematica Sinica 2(2): 123–136.Google Scholar
5. Chen, M.F. (1992). From Markov Chains to Non-equilibrium Particle Systems. Singapore: World Scientific Publishing Co.Google Scholar
6. Guo, X.P. and Hernández-Lerma, O. (2009). Continuous-time Markov decision processes . In Rozovskii, B. & Grimmett, G. (eds), Stochastic Modelling and Applied Probability, Vol. 62. Heidelberg: Springer-Verlag.Google Scholar
7. Guo, X.P., Hernández-Lerma, O., and Prieto–Rumeau, T. (2006). A survey of recent results on continuous-time Markov decision processes. TOP 14: 177–261.Google Scholar
8. Mertens, J.-F., Samuel-Cahn, E., and Zamir, S. (1978). Necessary and sufficient conditions for recurrence and transience of Markov chains in terms of inequalities. Journal of Applied Probability 15: 848–851.Google Scholar
9. Meyn, S.P. and Tweedie, R.L. (1995). Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Advances in Applied Probability 25: 518–548.Google Scholar
10. Norris, J.R. (2004). Markov Chains. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press.Google Scholar
11. Piunovskiy, A. and Zhang, Y. (2012). The transformation method for continuous-time Markov decision processes. Journal of Optimization Theory and Applications 154(2): 691–712.Google Scholar
12. Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games, ICP Advanced Texts in Mathematics. Vol. 5. London: Imperial College Press.Google Scholar
13. Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results. Annals of Operations Research doi:10.1007/s10479-012-1184-4.Google Scholar
14. Reuter, G.E.H. (1957). Denumerable Markov processes and the associated contraction semigroups on l . Acta Mathematica 97: 1–46.Google Scholar
15. Spieksma, F.M. (2013). Kolmogorov forward equation and explosiveness in countable state Markov processes. Annals of Operations Research doi: 10.1007/s10479-012-1262-7.Google Scholar
16. Tweedie, R.L. (1974). Some ergodic properties of the Feller minimal process. Journal of Mathematics 25(2): 485–493.Google Scholar