Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-28T15:24:57.059Z Has data issue: false hasContentIssue false
  • English
  • Français

Discrete time methods for simulating continuous time Markov chains

Published online by Cambridge University Press:  01 July 2016

Arie Hordijk ,
Donald L. Iglehart  and
Rolf Schassberger
Arie Hordijk
Affiliation:
Mathematisch Centrum, Amsterdam
Donald L. Iglehart
Affiliation:
Stanford University
Rolf Schassberger
Affiliation:
University of Calgary
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.

Keywords

MARKOV CHAINSREGENERATIVE PROCESSESSIMULATION METHODOLOGYVARIANCE REDUCTION METHODS
Type
Research Article
Information
Advances in Applied Probability , Volume 8 , Issue 4 , December 1976 , pp. 772 - 788
Copyright
Copyright © Applied Probability Trust 1976 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Chung, K. L. (1967) Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer-Verlag, Berlin.Google Scholar
[2] Çinlar, E. (1969) Markov renewal theory. Adv. Appl. Prob. 1, 123–187.CrossRefGoogle Scholar
[3] Crane, M. A. and Iglehart, D. L. (1974a) Simulating stable stochastic systems, I: General multi-server queues. J. Assoc. Comput. Mach. 21, 103–113.Google Scholar
[4] Crane, M. A. and Iglehart, D. L. (1974b) Simulating stable stochastic systems, II: Markov chains. J. Assoc. Comput. Mach. 21, 114–123.CrossRefGoogle Scholar
[5] Crane, M. A. and Iglehart, D. L. (1975) Simulating stable stochastic systems, III: Regenerative processes and discrete-event simulations. Operat. Res. 23, 33–45.CrossRefGoogle Scholar
[6] Iglehart, D. L. (1975) Simulating stable stochastic systems, V: comparison of ratio estimators. Naval Res. Logist. Quart. 22, 553–565.Google Scholar
[7] Jensen, A. (1953) Markoff chains as an aid in the study of Markoff processes. Skand. Aktuarietidskr. 36, 87–91.Google Scholar
[8] Jensen, A. and Kendall, D. (1971) Denumerable Markov processes with bounded generators: a routine for calculating pij (∞). J. Appl. Prob. 8, 423–427.Google Scholar
[9] Kemeny, J. G. and Snell, J. L. (1960) Finite Markov Chains. Van Nostrand, Princeton, N.J.Google Scholar
[10] Schassberger, R. (1968) Ein Wartesystem mit zwei parallelen Warteschlangen. Computing 3, 110–124.CrossRefGoogle Scholar