Several simple stochastic models are given for a finite closed system of individuals existing in clusters which may come together to form larger clusters which may in turn split up. Some of these models are analysed and compared in equilibrium. Several of the models fit into a general framework established and investigated by Whittle (1965a); it is shown that these models have an equilibrium solution of a particularly simple form, deduced by Whittle, if and only if the models are stochastically reversible. A normal approximation to two of the models in equilibrium is found to give the same mean value as a deterministic approximation.

A single-server queue with batch arrivals in a non-homogeneous Poisson process and with balking is studied with respect to the busy period, using supplementary variables. A system of integral equations is obtained on the base of which the transforms are expressed in series. For the homogeneous case, assuming finite waiting room, the solutions are obtained via Cramer's rule. This gives asymptotic expressions for the expectations for large arrival intensity. An efficiency measure giving the long run loss probability is given. For a special case contour integral representations are given as solutions.

A discrete-time Markov decision model with a denumerable set of states and unbounded costs is considered. It is shown that the optimality equation of dynamic programming along with some additional, easily checked, conditions may be used to establish the optimality or ∊ -optimality of policies with respect to the average expected cost criterion. The results are used to derive optimal policies in two queueing examples.

We consider a batch service queue which is controlled by switching the server on and off, and by controlling the batch size and timing of services. These batch sizes cannot exceed a fixed number Q, which we call the service capacity. Costs are charged for switching the server on and off, for serving customers and for holding them in the system. Viewing the system as a semi-Markov decision process, we show that the policies which minimize the expected continuously discounted cost and the expected cost per unit time over an infinite time horizon are of the following form: at a review point if the server is off, leave the server off until the number of customers x reaches an optimal level M, then turn the server on and serve min (x, Q) customers; and when the server is on, serve customers in batches of size min(x, Q) until the number of customers falls below an optimal level m(m ≦ M) and then turn the server off. An example for computing these optimal levels is also presented.

Criteria are established for a discrete-time Markov process {Xn}n≧0 in Rd to have strictly positive, respectively zero, probability of escaping to infinity. These criteria are mainly in terms of the mean displacement vectors μ(y) = E{Xn+1|Xn = y} – y, and are essentially such that they force a deterministic process w.p.1 to move off to infinity, respectively to return to a compact set infinitely often. As an application we determine of most two-dimensional birth and death processes with rates linearly dependent on the population, whether they can escape to infinity or not.

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.

Let Z(t) be the population size at time t in a general age-dependent branching process (as defined by Crump and Mode, or Jagers) in which the number N of offspring of a parent has expected value 1 (critical case). Assuming positivity and finiteness of the second moments of N, of the lifespan distribution and of the expected number of births per parent as a function of age (also assumed to be strongly non-lattice), the distribution of Z(t)/t conditioned on non-extinction at time t is asymptotically exponential. The main step in the proof is a comparison lemma for the probability generating functions of Z(t) and of the embedded generation process.

A stochastic model for a periodic screening program is presented in which the natural history of a chronic disease is assumed to follow a progressive path from a preclinical state to a clinical state. The sampling of preclinical state sojourn times by screening examinations generates bounds on the preclinical state recurrence times. The distribution of the bounded forward recurrence time is derived and used to obtain the distribution and mean of the lead time, and relationships for calculating the proportion of preclinical cases detected. These expressions are derived in terms of the preclinical state sojourn-time distribution and adjustible parameters important in the design of a periodic screening program.

This paper is concerned with a discrete Markov process related to an underlying weighted graph. With this graph we associate an urn containing p types of balls which correspond to the p vertices of the graph, and such that the number of balls of each type is proportional to the weight of the related vertex. A drawing scheme from the urn is defined which leads to Markovian non-homogeneous transition probabilities. Many kinds of weighted graphs are found to have a strong convergence property; this is that the number of balls drawn from the urn at the end of any single draw converges in probability to the total number of balls in the urn, as this tends to infinity.