Search

We consider a gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We show, under natural conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies that allow multiple simultaneous impulses, randomized selection of impulses with random effects, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.

By considering continuous-time Markov decision processes where decisions can be made at any time, we show in the case of M/M/1 queues with discounted costs that there exists a monotone optimal policy among all the regular policies.

Search Results

Refine search

Refine search

Actions for selected content:

2 results

On gradual-impulse control of continuous-time Markov decision processes with exponential utility

Adaptive control of M/M/1 queues—continuous-time Markov decision process approach

Search Results

Refine search

Refine search

Actions for selected content:

Save Search

2 results

On gradual-impulse control of continuous-time Markov decision processes with exponential utility

Adaptive control of M/M/1 queues—continuous-time Markov decision process approach