Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-28T09:18:00.690Z Has data issue: false hasContentIssue false
  • English
  • Français

Markov decision chains with unbounded costs and applications to the control of queues

Published online by Cambridge University Press:  01 July 2016

D. R. Robinson
D. R. Robinson*
Affiliation:
University of Sussex
Get access Rights & Permissions [Opens in a new window]

Abstract

A discrete-time Markov decision model with a denumerable set of states and unbounded costs is considered. It is shown that the optimality equation of dynamic programming along with some additional, easily checked, conditions may be used to establish the optimality or ∊ -optimality of policies with respect to the average expected cost criterion. The results are used to derive optimal policies in two queueing examples.

Keywords

MARKOV DECISION CHAINCONTROLLED QUEUEMINIMUM AVERAGE EXPECTED COST
Type
Research Article
Information
Advances in Applied Probability , Volume 8 , Issue 1 , March 1976 , pp. 159 - 176
Copyright
Copyright © Applied Probability Trust 1976 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bather, J. A. (1973) Optimal decision procedures for finite Markov chains. Part I: Examples. Adv. Appl. Prob. 5, 328–339. Part II: Communicating systems. Adv. Appl. Prob. 5, 521–540. Part III: General convex systems. Adv. Appl. Prob. 5, 541–553.Google Scholar
[2] Derman, C. (1966) Denumerable state Markovian decision processes-average cost criterion. Ann. Math. Statist. 37, 1545–1554.Google Scholar
[3] Derman, C. and Veinott, A. F. Jr. (1967) A solution to a countable system of equations arising in Markovian decision processes. Ann. Math. Statist. 38, 582–584.CrossRefGoogle Scholar
[4] Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematical Centre Tracts, No. 51, Amsterdam.Google Scholar
[5] Howard, R. A. (1960) Dynamic Programming and Markov Processes. M.I.T. Press, Cambridge, Mass. Google Scholar
[6] Jaiswal, N. K. (1968) Priority Queues. Academic Press, New York.Google Scholar
[7] Lippman, S. A. (1973) Semi-Markov decision processes with unbounded rewards. Management Sci. 7, 717–731.CrossRefGoogle Scholar