Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-09T16:07:03.747Z Has data issue: false hasContentIssue false
  • English
  • Français

Multiscale Stochastic Approximation for Parametric Optimization of Hidden Markov Models

Published online by Cambridge University Press:  27 July 2009

Shalabh Bhatnagar  and
Vivek S. Borkar
Shalabh Bhatnagar
Affiliation:
Department of Electrical Engineering, Indian Institute of Science, Bangalore 560 012, India
Vivek S. Borkar
Affiliation:
Department of Computer Science and Automation, Indian Institute of Science, Bangalore 560 012, India
Get access Rights & Permissions [Opens in a new window]

Abstract

A two–time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to “infinitesimal perturbation analysis.” Its convergence is analyzed, and a queueing example is presented.

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 11 , Issue 4 , October 1997 , pp. 509 - 522
Copyright
Copyright © Cambridge University Press 1997

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Borkar, V.S. (1990). The Kumar-Becker-Lin scheme revisited. Journal of Optimization Theory and Application 66: 287–309.CrossRefGoogle Scholar
2.Borkar, V.S. (1993). On white noise representations in stochastic realization theory. SIAM Journal on Control and Optimization 31: 1093–1102.CrossRefGoogle Scholar
3.Borkar, V.S. (1995). Probability theory: An advanced course. New York: Springer-Verlag.CrossRefGoogle Scholar
4.Borkar, V.S. & Phansalkar, V.V. (1994). Managing inter processor delays in distributed recursive algorithms. Sadhana 19(6): 995–1003.CrossRefGoogle Scholar
5.Chong, E.K.P. & Ramadge, P.J. (1993). Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM Journal on Control and Optimization 31(3): 698–732.CrossRefGoogle Scholar
6.Chong, E.K.P. & Ramadge, P.J. (1994). Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Transactions on Automated Control 39(7): 1400–1410.CrossRefGoogle Scholar
7.Dai, L. & Ho, Y.-C. (1995). Structural infinitesimal perturbation analysis (SIPA) for derivative estimation of discrete-event dynamic systems. IEEE Transactions on Automatic Control 40(7): 1154–1166.Google Scholar
8.Gelfand, S.B. & Mitter, S.K. (1992). Recursive stochastic algorithms for global optimization in Rd. SIAM Journal on Control and Optimization 29: 999–1018.CrossRefGoogle Scholar
9.Hirsch, M.W. (1987). Convergent activation dynamics in continuous time networks. Neural Networks 2: 331–349.CrossRefGoogle Scholar
10.Ho, Y.-C. & Cao, X.-R. (1991). Perturbation analysis of discrete event dynamical systems. Boston: Kluwer.CrossRefGoogle Scholar
11.Kushner, H.J. & Clark, D.S. (1978). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer-Verlag.CrossRefGoogle Scholar
12.Kushner, H.J. & Vazquez-Abad, F.J. (1994). Stochastic approximation methods for systems of interest over an infinite horizon. LCDS 94–4: 1–56.Google Scholar
13.Neveu, J. (1975). Discrete parameter martingales. Amsterdam: North-Holland.Google Scholar
14.Parthasarathy, K.R. (1967). Probability on metric spaces. New York: Academic Press.Google Scholar