The age structure of a large, unisexual, closed population is described here by a vector of the proportions in each age class. Non-negative matrices of age-specific birth and death rates, called Leslie matrices, map the age structure at one point in discrete time into the age structure at the next. If the sequence of Leslie matrices applied to a population is a sample path of an ergodic Markov chain, then: (i) the joint process consisting of the age structure vector and the Leslie matrix which produced that age structure is a Markov chain with explicit transition function; (ii) the joint distribution of age structure and Leslie matrix becomes independent of initial age structure and of the initial distribution of the Leslie matrix after a long time; (iii) when the Markov chain governing the Leslie matrix is homogeneous, the joint distribution in (ii) approaches a limit which may be easily calculated as the solution of a renewal equation. A numerical example will be given in Cohen (1977).

A stationary sequence of random variables with exponential marginal distributions and the correlation structure of an ARMA (1, 1) process is defined. The process is formed as a random linear combination of i.i.d. exponential random variables and is very simple to generate on a computer. Moments and joint distributions for the sequence are obtained, as well as limiting properties of sums of the random variables and of the point process whose intervals have the EARMA (1, 1) structure.

Grade and age structures in manpower systems are often far from ideal. This fact raises the question of how the flows of people — and particularly the recruitment flow — should be controlled in order to attain and maintain a more desirable structure. The problem has received considerable attention from a deterministic point of view. This paper adopts a stochastic approach to the study of maintainability and shows, among other things, that the problem is more subtle than the deterministic analysis suggests.

Let R(t) be the covariance function of the stationary virtual waiting-time process of a stable M/G/1 queue. It is proven that if R(t) exists, i.e., if the service-times have a finite third moment, then R(t) is positive and convex on [0, ∞), with an absolutely continuous derivative R’ and a bounded, non-negative second derivative R″. Also, and R″ cannot be chosen monotone. Contrary to a finding by Beneš [1] it is proven that if and only if the service-times have a finite fourth moment.

The problem of scheduling items for service with random service times is formulated as an optimal control problem. Pontryagin's maximum principle is used to determine the optimal schedule in certain cases.

An occupation measure describes the expected amount of time a stochastic process spends in different parts of its state space prior to a given random time. It is shown that a basic identity involving occupation measures provides a unified approach to a variety of moment identities for Markov chains, and some connections with potential theory are made.

We consider several problems in the theory of first-passage percolation on the two-dimensional integer lattice. Our results include: (i) a mean ergodic theorem for the first-passage time from (0,0) to the line x = n; (ii) a proof that the time constant is zero when the atom at zero of the underlying distribution exceeds C, the critical percolation probability for the square lattice; (iii) a proof of the a.s. existence of routes for the unrestricted first-passage processes; (iv) a.s. and mean ergodic theorems for a class of reach processes; (v) continuity results for the time constant as a functional of the underlying distribution.

Let X(t) be the virtual waiting-time process of a stable M/G/1 queue. Let R(t) be the covariance function of the stationary process X(t), B(t) the busy-period distribution of X(t); and let E(t) = P{X(t) = 0|X(0) = 0}.

For X(t) some heavy-traffic results are given, among which are limiting expressions for R(t) and its derivatives and for B(t) and E(t).

These results are used to find the covariance function of stationary Brownian motion on [0, ∞).

The systems considered are single-server, though the theory has wider application to models of adaptive queueing systems. Arrival and service mechanisms are governed by state (n)-dependent mean arrival and service rates λn and µn. It is assumed that the choice of λn and µn leads to a stable regime. Formulae are sought that provide easy means of computing statistics of effectiveness of systems. A measure of traffic intensity is first defined in terms of ‘effective’ service time and inter-arrival intervals. It is shown that the latter have a renewal type connection with appropriately defined mean effective arrival and service rates λ∗ and µ∗ and that in consequence the ratio λ∗/µ∗ is the traffic intensity, equal moreover to where is the stable probability of an empty system, consistent with other systems. It is also shown that for first come, first served discipline the equivalent of Little's formula holds, where and are the mean waiting time of an arrival and mean system size at an arbitrary epoch. In addition it appears that stable regime output intervals are statistically identical with effective inter-arrival intervals. Symmetrical moment formulae of arbitrary order are derived algebraically for effective inter-arrival and service intervals, for waiting time, for busy period and for output.

We consider a controlled diffusion process, the description of which depends on an unknown parameter α, and investigate the following control policy. To each α an optimal stationary control is associated. α is estimated recurrently from the trajectory by Bayes' method, and the optimal stationary control corresponding to the estimate is used. We establish the consistency of the estimate, and present asymptotic properties of the criterion function. They follow from the central limit theorem, from the law of large numbers and from the law of the iterated logarithm for local martingales.

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

This paper is mainly concerned with the asymptotic theory of maximum likelihood estimation for continuous-time stochastic processes. The role of martingale limit theory in this theory is developed. Some analogues of classical statistical concepts and quantities are also suggested. Various examples that illustrate parts of the theory are worked through, producing new results in some cases. The role of diffusion approximations in estimation is also explored.

A new method for obtaining spectral-like representations for a large class of non-stationary random processes is formulated. For a wide sense stationary process X(t) in continuous-time the spectral representation is generated by a self-adjoint operator H such that X(t)= eiHt X(0). Extending certain recently established operator identities for wide sense stationary processes, it is shown that similar operators exist for classes of non-stationary processes. The representation generated by such an operator has the form and it shares some of the properties of the wide sense stationary spectral representation: it is dual in a precisely defined sense to the time domain representation of X(t). There exists a class L of linear transformations of X(t) such that for G ∈ L for some function g determined by G.