Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-28T18:10:17.888Z Has data issue: false hasContentIssue false
  • English
  • Français

A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices

Published online by Cambridge University Press:  14 July 2016

A. Federgruen ,
A. Hordijk  and
H. C. Tijms
A. Federgruen
Affiliation:
Mathematisch Centrum, Amsterdam
A. Hordijk
Affiliation:
Rijksuniversiteit Leiden
H. C. Tijms
Affiliation:
Vrije Universiteit, Amsterdam
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we consider a set of denumerable stochastic matrices where the parameter set is a compact metric space. We give a number of simultaneous recurrence conditions on the stochastic matrices and establish equivalences between these conditions. The results obtained generalize corresponding results in Markov chain theory to a considerable extent and have applications in stochastic control problems.

Keywords

COMPACT METRIC SET OF DENUMBERABLE STOCHASTIC MATRICESSIMULTANEOUS RECURRENCE CONDITIONSDOEBLIN CONDITIONSCRAMBLING CONDITIONQUASI-COMPACTNESS CONDITIONEQUIVALENCES
Type
Short Communications
Information
Journal of Applied Probability , Volume 15 , Issue 4 , December 1978 , pp. 842 - 847
Copyright
Copyright © Applied Probability Trust 1978 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Doob, J. L. (1953) Stochastic Processes. Wiley, New York.Google Scholar
[2] Federgruen, A. and Tijms, H. C. (1978) The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms. J. Appl. Prob. 15, 356–373.CrossRefGoogle Scholar
[3] Federgruen, A., Hordijk, A. and Tijms, H. C. (1978) Recurrence conditions in denumer able state Markov decision processes In Dynamic Programming and its Applications , ed. Puterman, M. L. Academic Press, New York.Google Scholar
[4] Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematical Centre Tract No. 51, Mathematisch Centrum, Amsterdam.Google Scholar
[5] Huang, C., Isaacson, D. and Vinograde, B. (1976) The rate of convergence of certain nonhomogeneous Markov chains. Z. Wahrscheinlichkeitsth. 35, 141–146.CrossRefGoogle Scholar
[6] Isaacson, D. and Madsen, R. (1974) Positive columns for stochastic matrices. J. Appl. Prob. 11, 829–835.CrossRefGoogle Scholar
[7] Neveu, J. (1965) Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco.Google Scholar
[8] Rudin, W. (1964) Principles of Mathematical Analysis , 2nd edn. McGraw-Hill, New York.Google Scholar