A birth–death process {x(t): t ≥ 0} with state space the set of non-negative integers is said to be stochastically increasing (decreasing) on the interval (t1, t2) if Pr {x(t) > i} is increasing (decreasing) with t on (t1, t2) for all i = 0, 1, 2, ···. We study the problem of finding a necessary and sufficient condition for a birth–death process with general initial state probabilities to be stochastically monotone on an interval. Concrete results are obtained when the initial distribution vector of the process is a unit vector. Fundamental in the analysis, and of independent interest, is the concept of dual birth–death processes.

The multitype continuous-time Markov branching process has many biological applications where the environmental factors vary in a periodic manner. Circadian or diurnal rhythms in cell kinetics are an important example. It is shown that in the supercritical positively regular case the proportions of individuals of various types converge in probability to a non-random periodic vector, independent of the initial conditions, while the absolute numbers of individuals of various types converge in probability to that vector multiplied by a random variable whose distribution depends on the initial conditions. It is noted that the proofs are straightforward extensions of the well-known results for a constant environment.

We study clustering (‘polymerisation’) processes under the assumptions that new bonds may or may not form within an existing cluster. We show that both models exhibit criticality (‘gelation’) at the same point, and investigate statistics of polymer size and structure beyond criticality in the first case. Gelation may or may not be accompanied by critical effects at unit level; we examine a prototype of each case: the cosh model and the Poisson model.

We consider a polymerisation process stratified in that space is divided into regions, between which migration occurs, but with bonding occurring only within a region. In the case of a process whose specification is spatially homogeneous, criticality (gelation) is then easily detectable as the point at which statistical equidistribution over regions becomes unstable. Stratification does import a new feature, however, in that the equipartition solution can become metastable below criticality; local fluctuations of density can induce ‘gelational collapse’ at a density below the critical value. We derive also detailed results for the inhomogeneous case, both below and above criticality. Statistics of polymer size and structure are also easily determined in the stratified case, although one can locate the critical point without recourse to these. Finally, one can to a large extent treat the case in which inter- and intrapolymer reaction rates differ, and show that such difference affects the onset of metastability rather than of instability.

The solution for the multitype version of the polymerisation process treated in earlier papers is examined, and found to present several new features, because of the necessarily complex integration path, and because of the greater number and variety of potential singularities. Cases of ‘Poisson interaction’ are particularly amenable to treatment, however, and we examine two particular cases of interest in Sections 5 and 6. One is a version of the Ising model, which shows phase transition and its character very simply. The other is a version of the farmer–trader model considered previously, which now manifests a number of features, including a discontinuous transition between agrarian and mercantile economies. A stratified Poisson-interaction model is examined in Section 7. The roles of the various singularities are clarified, and a behaviour analogous to that of the single-type case confirmed.

A stationary Markovian decision model is considered with general state and action spaces where the transition probabilities are weakened to be bounded transition measures (this is useful for many applications). New and improved bounds are given for the optimal value of stationary problems with a large planning horizon if either only a few steps of iteration are carried out or, in addition, a solution of the infinite-stage problem is known. Similar estimates are obtained for the quality of policies which are composed of nearly optimal decisions from the first few steps or from the infinite-stage solution.

Given a finite number of different experiments with unknown probabilities p1, p2, ···, pk of success, the multi-armed bandit problem is concerned with maximising the expected number of successes in a sequence of trials. There are many policies which ensure that the proportion of successes converges to p = max (p1, p2, ···, pk), in the long run. This property is established for a class of decision procedures which rely on randomisation, at each stage, in selecting the experiment for the next trial. Further, it is suggested that some of these procedures might perform well over any finite sequence of trials.