Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-26T09:31:07.361Z Has data issue: false hasContentIssue false
  • English
  • Français

Strong consistency of a modified maximum likelihood estimator for controlled Markov chains

Published online by Cambridge University Press:  14 July 2016

Bharat Doshi  and
Steven E. Shreve
Bharat Doshi*
Affiliation:
Bell Laboratories
Steven E. Shreve*
Affiliation:
Carnegie–Mellon University
*
∗Postal address: HP1B323, Holmdel, NJ 07733, U.S.A. Research carried out when the author was at Rutgers University.
∗∗Postal address: Department of Mathematics, Carnegie-Mellon University, Pittsburgh, PA 15213, U.S.A.
Get access Rights & Permissions [Opens in a new window]

Abstract

A controlled Markov chain with finite state space has transition probabilities which depend on an unknown parameter α lying in a known finite set A. For each α, a stationary control law ϕ α is given. This paper develops a control scheme whereby at each stage t a parameter α t is chosen at random from among those parameters which nearly maximize the log likelihood function, and the control ut is chosen according to the control law ϕ αt. It is proved that this algorithm leads to identification of the true α under conditions weaker than any previously considered.

Keywords

CONTROLLED MARKOV CHAINIDENTIFICATION
Type
Research Papers
Information
Journal of Applied Probability , Volume 17 , Issue 3 , September 1980 , pp. 726 - 734
Copyright
Copyright © Applied Probability Trust 1980 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Research sponsored in part by the Air Force Office of Scientific Research (AFSC), United States Air Force, under Contract F-49620–79-C-0165.

References

[1] Borkar, V. and Varaiya, P. (1979) Adaptive control of Markov chains, I: Finite parameter set. IEEE Trans. Auto. Control 24, 953–957.Google Scholar
[2] Loève, M. (1960) Probability Theory Van Nostrand, Princeton, NJ.Google Scholar
[3] Mandl, P. (1974) Estimation and control in Markov chains. Adv. Appl. Prob. 6, 40–60.CrossRefGoogle Scholar