Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-27T08:43:35.289Z Has data issue: false hasContentIssue false
  • English
  • Français

Iterative Bounds on the Equilibrium Distribution of a Finite Markov Chain

Published online by Cambridge University Press:  27 July 2009

Jan Van Der Wal  and
Paul J. Schweitzer
Jan Van Der Wal
Affiliation:
Department of Mathematics and Computing ScienceUniversity of Technology Eindhoven, The Netherlands
Paul J. Schweitzer
Affiliation:
William E. Simon Graduate School of Business Administration University of Rochester Rochester, New York
Get access Rights & Permissions [Opens in a new window]

Abstract

This article presents a new iterative method for computing the equilibrium distribution of a finite Markov chain, which has the significant advantage of providing good upper and lower bounds for the equilibrium probabilities. The method approximates the expected number of visits to each state between two successive visits to a given reference state. Numerical examples indicate that the performance of this method is quite good.

Type
Articles
Information
Probability in the Engineering and Informational Sciences , Volume 1 , Issue 1 , January 1987 , pp. 117 - 131
Copyright
Copyright © Cambridge University Press 1987

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aitken, A. C. (1950). On the iterative solution of a system of linear equations. Proc. Royal Soc. Edinburgh, A 63: 52–80.Google Scholar
Albrecht, J. (1962). Fehlerschranken und Konvergenrbeschlennigung bei einer monotonen oder alternierenden Iterationsfolge. Num. Mathematick 4: 196–208.CrossRefGoogle Scholar
Berman, A. and Plemmons, R. J. (1979). Nonnegative Matrices in the Mathematical Sciences, Academic Press, New York.Google Scholar
Denardo, E. V. (1967). Contraction mappings in the theory underlying dynamic programming. SIAM Review 9: 165–177.CrossRefGoogle Scholar
Doob, J. L. (1953). Stochastic Processes, Wiley, New York, p. 173.Google Scholar
Jennings, A. and Stewart, W. J. (1975). Simultaneous iteration for partial eigensolution of real matrices. J. Inst. Math. Appl. 15: 351–361.CrossRefGoogle Scholar
Kemeny, J. G. and Snell, J. L. (1976). Finite Markov Chains, Springer-Verlag, New York, p. 46.Google Scholar
Pease, M. C. (1965). Methods of Matrix Algebra, Academic Press, New York.Google Scholar
Porteus, E. L. (1971). Some bounds for discounted sequential decision processes. Mgmt. Science 18: 7&11.CrossRefGoogle Scholar
Porteus, E. L. and Totten, J. C. (1978). Accelerated computation of the expected discounted return in a Markov chain. Oper. Res. 26: 350–358.CrossRefGoogle Scholar
Schellhaas, H. (1974). Zur Extrapolation in Markoffschen Entscheidungsmodellen mit Discontierung. Zeitschr. F. Oper. Res. 18: 91–104.Google Scholar
Schweitzer, P. J. (1971). Iterative solution of the functional equations of undiscounted Markov renewal programming. J. Math. Anal. Appl. 34: 495–501.CrossRefGoogle Scholar
Schweitzer, P. J. (1985). Bounds on the fixed point of a monotone contraction operator. To appear in. J. Math. Anal. Appl.Google Scholar
Seneta, E. (1980). Nonnegative Matrices and Markov Chains, 2nd edition, Springer-Verlag, Berlin, West Germany.Google Scholar
Varga, R. (1962). Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, New Jersey, p. 70, 75, 281.Google Scholar
Wal, J. van der and Schweitzer, P. J. (1985). Iterative bounds on the equilibrium distribution of a finite Markov chain. Memorandum COSOR 85–08. Department of Mathematics and Computer Science, University of Technology, Eindhoven, The Netherlands.Google Scholar
Waldmann, K. H. (1986). Bounds to the distribution of the run length in general quality-control schemes, Statiskische Hefte 27: 37–56.CrossRefGoogle Scholar