An abstract gambler's problem is formulated in a continuous-time setting and analogues are proved for some of the discrete-time results of Dubins and Savage in their book How to Gamble if You Must. Applications are made to problems of controlling a Brownian motion process.

The problem discussed is that of controlling optimally the release of water from a finite dam, when the release rate may vary continuously and the optimality is defined in terms of a cost structure imposed on the operation of the dam. A diffusion model is suggested and by considering a family of plausible output policies, the control problem is reduced to the solution of a free boundary problem associated with a certain partial differential equation. A set of necessary conditions for the optimal choice of these boundaries is established and a method of solution is suggested. By using this method, together with well-established computational techniques, numerical solutions are obtained. These numerical solutions indicate that this optimal policy does not result in very much improvement over a much simpler policy where the output rate is constrained to take only two values.

We obtain results connecting the distributions of the random variables Z1 and W in the supercritical Galton-Watson process. For example, if a > 1, and converge or diverge together, and regular variation of the tail of one of Z1, W with non-integer exponent α > 1 is equivalent to regular variation of the tail of the other.

For a Markov process in discrete time, both (a) the time-dependent behaviour of the process at time t, and (b) the behaviour of moments (if finite) of the process, whether or not stationary, as a function of time-lag t, are determined by solutions to a single equation, the key equation. The state space of the process is required only to be both a measure space, and a locally convex topological vector space, which may have infinite dimension. The asymptotic behaviour of both (a) and (b), for large times t, is shown to be (nearly) the same, being proportional to tξe−βt, where the exponent β is calculable from an eigenvalue of a linear operator, describing the Markov process. For a large class of Markov processes, the spectrum of this operator is shown to consist of discrete eigenvalues only. Queueing applications are discussed.

We illustrate a technique for computing certain integrals that arise in probability theory by giving a new derivation of a formula of Kingman. This formula contains the joint distribution of the processes F(t) = inf {s: X(t + s) = b} and B(t) = inf{s: X(t - s) = b} where X is a time homogeneous, continuous parameter, Markov process and b is a fixed point in its state space. We then extend this formula to the situation in which b is replaced by a finite set {b1, …, bn}.

A compact convex set in RN is Steiner if it is a finite Minkowski sum of line segments, or a limit of such finite sums, and then satisfies an extension of the Steiner formula. With each Poisson hyperplane stationary process A is uniquely associated a Steiner set M, and for any linear variety V, the Steiner set associated with is the projection of M on V. The density of the order k network Ak (i.e., the set of the intersections of k hyperplanes belonging to A) is linked with simple geometrical properties of M. In the isotropic case, the expression of the covariance measures associated with Ak is derived and compared with the analogous results obtained for (N — k)-dimensional Poisson flats.

Policies of immunising the susceptibles in the general stochastic epidemic are considered. With costs assigned to the infection and immunisation of susceptibles, an optimal policy is found which at any stage minimises the expected future cost. Optimal policies are also found for the general deterministic epidemic and for an epidemic with carriers model. The three policies are compared and numerical examples are provided. An optimal policy is described in the situation when both isolation of infectives and immunisation of susceptibles are allowed.

This paper develops a comparison method for critical branching processes. The method is applied to prove the exponential limit law for the multi-type age-dependent process under second moment conditions.

Dynamic programming is used to solve some simple gambling models. In particular we consider the situation where an individual may bet any integral amount not greater than his fortune and he will win this amount with probability p or lose it with probability 1 — p. It is shown that if p ≧ ½ then the timid strategy (always bet one dollar) both maximizes the probability of ever reaching any preassigned fortune, and also stochastically maximizes the time until the bettor becomes broke. Also, if p ≦ ½ then the timid strategy while not stochastically maximizing the playing time does maximize the expected playing time. We also consider the same model but with the additional structure that the bettor need not gamble but may instead elect to work for some period of time. His goal is to minimize the expected time until his fortune reaches some preassigned goal. We show that if p ≦ ½ then (i) always working is optimal, and (ii) among those strategies that only allow working when the bettor is broke it is the bold strategy that is optimal

A single realization {Zn, 0 ≦n≦N + 1} of a supercritical Galton-Watson process (so called) is considered and it is required to estimate the variance of the offspring distribution. A prospective estimator is proposed, where , and is shown to be strongly consistent on the non-extinction set. A central limit result and an iterated logarithm result are provided to give information on the rate of convergence of the estimator. It is also shown that the estimation results are robust in the sense that they continue to apply unchanged in the case where immigration occurs. Martingale limit theory is employed at each stage in obtaining the limit results.

This paper deals with the computation of survival probabilities and extinction times for multitype positively regular branching processes. If all of the generating functions of the offspring distributions are of the linear fractional form and have the same denominator, explicit expressions may be obtained for all of their iterates. It is then possible to obtain formulae for survival probabilities and bounds on the mean time to extinction, given extinction, of a line descended from a single individual. If there are two types and the offspring distributions are bivariate Poisson, their generating functions may be bounded by linear fractional generating functions. It is then possible to compute upper and lower bounds on mean times to extinction, given extinction, and this is done for some special cases.

It is shown that two different population models are equivalent to each other when a certain marginal stationary distribution is considered.

It is demonstrated for the non-critical and the explosive cases of the simple Bienaymé-Galton-Watson (B. G. W.) process (with and without immigration) that there exists a natural and intimate connection between regularly varying function theory and the asymptotic structure of the limit laws and corresponding norming constants. A similar fact had been demonstrated in connection with their invariant measures in [22]. This earlier study is complemented here by a similar analysis of the process where immigration occurs only at points of “emptiness” of the B. G. W. process.

In an earlier paper (Renewal theory in two dimensions: Basic results) the author developed a unified theory for the study of bivariate renewal processes. In contrast to this aforementioned work where explicit expressions were obtained, we develop some asymptotic results concerning the joint distribution of the bivariate renewal counting process (Nx(1), Ny(2)), the distribution of the two-dimensional renewal counting process Nx,y and the two-dimensional renewal function &Nx,y. A by-product of the investigation is the study of the distribution and moments of the minimum of two correlated normal random variables. A comprehensive bibliography on multi-dimensional renewal theory is also appended.

As an alternative to conventional discrete time models for stochastic processes that fluctuate within the sampling interval, we propose difference equations containing non-integral lags. We discuss the problems of stability, identification and estimation, for which an approximate model is needed. Least squares applied to an approximate Fourier-transformed model yields estimators of the coefficients that are consistent with respect to the true model under some conditions. The conditions are weak when the model contains predetermined variables that obey an “aliasing condition”; estimators of the lags as well as coefficients can then be found that are consistent, efficient and satisfy a central limit theorem. Optimal estimators for stochastic difference-differential equations are also available.

Many physical problems are described by stochastic processes with stationary increments. We present a general description of such processes. In particular we give an expression of a process in terms of its increments and we show that there are two classes of processes: diffusion and asymptotically stationary. Moreover, we show that the nth increments are given by a linear filtering of an arbitrary stationary process.