Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-27T07:33:29.511Z Has data issue: false hasContentIssue false
  • English
  • Français

Conditions Under Which a Markov Chain Converges to its Steady State in Finite Time

Published online by Cambridge University Press:  27 July 2009

Peter W. Glynn  and
Donald L. Iglehart
Peter W. Glynn
Affiliation:
Department of Operations Research Stanford University, Stanford, CA 94305
Donald L. Iglehart
Affiliation:
Department of Operations Research Stanford University, Stanford, California 94305
Get access Rights & Permissions [Opens in a new window]

Abstract

Analysis of the initial transient problem of Monte Carlo steady-state simulation motivates the following question for Markov chains: when does there exist a deterministic Tsuch that P{X(T) = y|(0) = x} = ®(y), where ρ is the stationary distribution of X? We show that this can essentially never happen for a continuous-time Markov chain; in discrete time, such processes are i.i.d. provided the transition matrix is diagonalizable.

Type
Articles
Information
Probability in the Engineering and Informational Sciences , Volume 2 , Issue 3 , July 1988 , pp. 377 - 382
Copyright
Copyright © Cambridge University Press 1988

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Cinlar, E. (1975). Introduction to stochastic processes. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Ethier, S.N. & Kurtz, T.G. (1986). Markov processes: Characterization and convergence. New York: John Wiley.CrossRefGoogle Scholar
Feller, W. (1968). An introduction to probability theory and its applications, Vol. 1. New York: John Wiley.Google Scholar
Lancaster, P. (1969). Theory of matrices. New York: Academic Press.Google Scholar
Schiuben, L. (1983). Confidence interval estimation using standardized time series. Operations Research 31: 1090–1108.Google Scholar
Subelman, E.J. (1976). On the class of Markov chains with finite convergence time. Stochastic Processes Applications 4: 253–259.CrossRefGoogle Scholar