Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-28T00:04:00.397Z Has data issue: false hasContentIssue false
  • English
  • Français

Bounds and Approximations for the Transient Behavior of Continuous-Time Markov Chains

Published online by Cambridge University Press:  27 July 2009

Bok Sik Yoon  and
J. George Shanthikumar
Bok Sik Yoon
Affiliation:
Department of Industrial Engineering and Operations Research University of California Berkeley, California, 94720
J. George Shanthikumar
Affiliation:
School of Business Administration University of California Berkeley, California, 94720
Get access Rights & Permissions [Opens in a new window]

Extract

Discretization is a simple, yet powerful tool in obtaining time-dependent probability distribution of continuous-time Markov chains. One of the most commonly used approaches is uniformization. A recent addition to such approaches is an external uniformization technique. In this paper, we briefly review these different approaches, propose some new approaches, and discuss their performances based on theoretical bounds and empirical computational results. A simple method to get lower and upper bounds for first passage time distribution is also proposed.

Type
Articles
Information
Probability in the Engineering and Informational Sciences , Volume 3 , Issue 2 , April 1989 , pp. 175 - 198
Copyright
Copyright © Cambridge University Press 1989

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Cinlar, E. (1975). Introduction to stochastic processes. Englewood Cliffs, New Jersey: Prentice Hall.Google Scholar
Van, Dijk N.M. (1988). A note on error bounds for approximating transition probabilities in continuous-time Markov chains. Probability in the Engineering and Informational Sciences 2: 471–474.Google Scholar
Grassmann, W. (1977). Transient solutions in Markovian queues. European Journal of Operations Research 1: 396–402.CrossRefGoogle Scholar
Keilson, J. (1979). Markov chain models–rarity and exponentiality. New York: Springer-Verlag.CrossRefGoogle Scholar
Ross, S.M. (1983). Stochastic processes. New York: Wiley.Google Scholar
Ross, S.M. (1987). Approximating transition probabilities and mean occupation times in continuous-time Markov chains. Probability in the Engineering and Informational Sciences 1: 251–264.CrossRefGoogle Scholar
Shaked, M. & Shanthikumar, J.G. (1988). Temporal stochastic convexity and concavity. Stochastic Processes and their Applications 27: 1–20.CrossRefGoogle Scholar
Stoyan, D. (1983). Comparison methods for queues and other stochastic models. New York: Wiley.Google Scholar
Sumita, U. & Shanthikumar, J.G. (1986). A software reliability model with multiple-error introduction and removal. IEEE Transactions on Reliability R-35: 459–462.CrossRefGoogle Scholar
Yoon, B.S. (1988). Approximations for the transient behavior of stochastic processes: discrelization and uniformization. Ph.D. Dissertation, University of California, Berkeley, California.Google Scholar