Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-18T17:49:46.107Z Has data issue: false hasContentIssue false
  • English
  • Français

Generalized semi-Markov decision processes

Published online by Cambridge University Press:  14 July 2016

Bharat T. Doshi
Bharat T. Doshi*
Affiliation:
Rutgers University
*
∗ Postal address: Department of Statistics, Hill Center for Mathematical Sciences, Rutgers University, New Brunswick NJ 08903, U.S.A.
Get access Rights & Permissions [Opens in a new window]

Abstract

Various authors have derived the necessary and sufficient conditions for optimality in semi-Markov decision processes in which the state remains constant between jumps. In this paper similar results are presented for a generalized semi-Markov decision process in which the state varies between jumps according to a Markov process with continuous sample paths. These results are specialized to a general storage model and an application to the service rate control in a GI/G/1 queue is indicated.

Keywords

GENERALIZED SEMI-MARKOV DECISION PROCESSESDISCOUNTED COSTNECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITYSTORAGE MODELGI/G/1 QUEUE
Type
Research Papers
Information
Journal of Applied Probability , Volume 16 , Issue 3 , September 1979 , pp. 618 - 630
Copyright
Copyright © Applied Probability Trust 1979 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Research partly supported by a Summer Research Fellowship at Rutgers University.

References

[1] Chitgopekar, S. S. (1969) Continuous time Markovian sequential control process. SIAM J. Control 7, 367–389.Google Scholar
[2] Doshi, B. T. (1976) Continuous time control of Markov processes on an arbitrary state space: discounted rewards. Ann. Statist. 4, 1219–1235.Google Scholar
[3] Doshi, B. T. (1976) Continuous time control of Markov processes on an arbitrary state space: average return criterion. Stoch. Proc. Appl. 4, 55–77.Google Scholar
[4] Doshi, B. T. (1976) Markov decision processes with both continuous and lump costs. Technical Report, Rutgers University.Google Scholar
[5] Doshi, B. T. (1978) Optimal control of the service rate in an M/G/1 queue. Adv. Appl. Prob. 10, 682–701.Google Scholar
[6] Doshi, B. T. (1978) Generalized semi-Markov decision processes. Technical Report, Rutgers University.Google Scholar
[7] Dynkin, E. B. (1965) Markov Processes I. Springer-Verlag, New York.Google Scholar
[8] Jewell, W. S. (1963) Markov renewal programming I, II. Opns Res. 11, 949–971.Google Scholar
[9] Mitchell, W. (1973) Optimal service selection rate selection in an M/G/1 queue. SIAM J. Appl. Math. 24, 19–35.Google Scholar
[10] Morais, P. R. (1976) Optimal Control of a Storage System. Ph.D. Thesis, Northwestern University.Google Scholar
[11] Pliska, S. (1977) On a functional differential equation that arises in a Markov control problem. Technical Report, Northwestern University.Google Scholar
[12] Ross, S. (1970) Average cost semi-Markov decision processes. J. Appl. Prob. 7, 649–656.Google Scholar
[13] Ross, S. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.Google Scholar
[14] Stone, L. D. (1973) Necessary and sufficient conditions for optimal control of semi-Markov processes. SIAM J. Control 11, 187–201.Google Scholar