Computer simulations had suggested that the strategy that maximises the score on each turn in the dice game described by Roters (1998) may not be the optimal way to reach a given target in the shortest time. We give an analytical treatment, backed by numerical calculations, that finds the optimal strategy to reach such a target.

In this paper we consider an explicit solution of an optimal stopping problem arising in connection with a dice game. An optimal stopping rule and the maximum expected reward in this problem can easily be computed by means of the distributions involved and the specific rules of the game

Consider a decision-maker who is in charge of a number of activities. At each of a sequence of decision points in time, he selects—on the basis of their performance—the set of activities to be continued further and allocates his limited resources among them. Activities receiving larger allocations tend to improve their performance, while others receiving smaller allocations tend to deteriorate. We present a controlled random walk model for the progress of these activities. The problem of maximizing the net infinite horizon discounted return is formulated in the framework of Markov decision theory, and existence of optimal strategies established. It is shown that both the optimal selection and allocation strategies exhibit a ‘favoring the leaders' behavior. Finally, explicit solutions to certain special cases are obtained illustrating these results.

This paper considers the discrete time Markov decision model with a denumerable state space and finite action space. Under certain conditions it is proved that the minimal total expected cost for a planning horizon of n epochs minus n times the minimal long-run average expected cost per unit time has a finite limit as n → ∞ for each initial state.