Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2024-12-27T12:35:52.366Z Has data issue: false hasContentIssue false
  • English
  • Français

Occupation measures for Markov chains

Published online by Cambridge University Press:  01 July 2016

J. W. Pitman
J. W. Pitman*
Affiliation:
University of California, Berkeley
Get access Rights & Permissions [Opens in a new window]

Abstract

An occupation measure describes the expected amount of time a stochastic process spends in different parts of its state space prior to a given random time. It is shown that a basic identity involving occupation measures provides a unified approach to a variety of moment identities for Markov chains, and some connections with potential theory are made.

Keywords

markov chainsstopping timesoccupation timespotential theory
Type
Research Article
Information
Advances in Applied Probability , Volume 9 , Issue 1 , March 1977 , pp. 69 - 86
Copyright
Copyright © Applied Probability Trust 1977 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Blumenthal, R. M. and Getoor, R. K. (1968) Markov Processes and Potential Theory. Academic Press, New York and London.Google Scholar
[2] Chung, K. L. (1967) Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer, Berlin.Google Scholar
[3] Derman, C. (1954) A solution to a set of fundamental equations in Markov chains. Proc. Amer. Math. Soc. 5, 332–334.Google Scholar
[4] Dynkin, E. B. and Yushkevitch, A. A. (1969) Markov Processes; Theorems and Problems. Plenum Press, New York.CrossRefGoogle Scholar
[5] Feller, W. (1971) An Introduction to Probability Theory and its Applications, Vol. 2, 2nd edn. Wiley, New York.Google Scholar
[6] Freedman, D. (1971) Markov Chains. Holden Day, San Francisco.Google Scholar
[7] Itô, K. and McKean, H. P. (1965) Diffusion Processes and their Sample Paths. Springer, Berlin.Google Scholar
[8] Kemeny, J. G. and Snell, J. L. (1961) Potentials for denumerable Markov chains. J. Math. Anal. Appl. 3, 196–260.CrossRefGoogle Scholar
[9] Kemeny, J. G. and Snell, J. L. (1961) Finite continuous time Markov chains. Theor. Prob. Appl. 6, 110–115.Google Scholar
[10] Kemeny, J. G., Snell, J. L. and Knapp, A. W. (1966) Denumerable Markov Chains. Van Nostrand, Princeton.Google Scholar
[11] Lamperti, J. (1960) The first passage moments and the invariant measure of a Markov chain. Ann. Math. Statist. 31, 515–517.Google Scholar
[12] Meyer, P. A. (1966) Probability and Potentials. Blaisdell, Waltham, Mass. Google Scholar
[13] Neveu, J. (1964) Chaînes de Markov et théorie du potentiel. Ann. Fac. Sci. Clermond-Ferrand 3, 37–65.Google Scholar
[14] Neveu, J. (1970) Chaînes de Markov. Textos de Matematica, no. 19, Instituto de Matematica, Univ. Federal de Pernambuco.Google Scholar
[15] Pitman, J. W. (1974) An identity for stopping times of a Markov process. In Studies in Probability and Statistics, ed. Williams, E. J.. Jerusalem Academic Press.Google Scholar
[16] Pitman, J. W. (1974) Uniform rates of convergence for Markov chain transition probabilities. Z. Wahrscheinlichkeitsth. 29, 193–227.Google Scholar
[17] Riordan, J. (1958) An Introduction to Combinatorial Analysis. Wiley, New York.Google Scholar
[18] Walsh, J. and Weil, M. (1972) Representation de temps terminaux et applications aux fonctionelles additives et aux systèmes de Lévy. Ann. Sci. École Norm. Sup. (4) 5, 121–155.Google Scholar