It is shown that methods used by the author to approximate limit probabilities for Markov processes from their Q-matrices extend to semi-Markov processes. The limit probabilities for semi-Markov processes can be approximated using only truncations of the embedded Markov chain transition matrix and the vector of mean holding times.

The change with respect to time of the information in a population is studied, when the population is classified according to genetic type. The cases considered are (i) an infinite population classified at two loci, (ii) a large finite population classified at one locus, subject to sampling fluctuations and (iii) the same, subject to the effect of mutation.

The Jackson network under study receives batch arrivals at i.i.d. intervals and features Markovian routing and exponentially distributed service times. The system is shown to be stable, in the sense of not being overloaded, if and only if, for each node, the total arrival rate of external and internal customers is less than the service rate. The method of proof is of more general interest.

A discrete time, scalar, pursuit-evasion game is presented in which an evader, moving according to a stationary stochastic process, is continually being followed by a pursuer. Both players have perfect observations of the evader's positions, but the observations of the pursuer are subject to a time lag. It is assumed that the strategy of the evader can be represented as an infinite moving average, and that he is restricted by a velocity constraint. The pursuer is limited to strategies linear in his information, and the payoff is taken to be the mean-square distance between pursuer and evader. Under these conditions it is shown that the game does not have a value, and subsequently the lower and upper values and corresponding strategies are found.

The information flow in discrete Markov systems provides a method for determining that such a system is ergodic. Estimates are obtained for the information flow in some classes of Markov systems and using these estimates criteria for the ergodicity of the systems are established.

In many endemic areas the level of infection of malaria exhibits a regular cycle. A stochastic model is formulated to describe endemic malaria and is used to justify a set of deterministic equations. The model incorporates the properties peculiar to malaria, i.e., in endemic areas malaria is not usually fatal and an individual can be and usually is reinfected. The theory of small oscillations is applied to the deterministic equations, and under the assumption of stable populations of hosts and vectors an expression is obtained for the period of oscillation of the infection.

In this paper we consider maximum likelihood estimation of the rate constant for stochastic rth-order reactions based on observation of the level of the system at time t > 0. Conditions are found for which the waiting time until the nth. reaction event is asymptotically normal, as the initial number of molecules and the number of reaction events become large. This distributional result is used to derive an approximate estimator which is shown for the second-order case to be close to the exact maximum likelihood estimate over a wide range of percentage completion.

We present, discuss and prove an apparently unfamiliar result in the convergence of probability measures.

Consider an autoregressive-moving-average process of given order where it is known that a number of moving-average roots are of unit modulus. Such a situation might arise, for example, when a time series has been differenced to induce stationarity by removing a non-stationary polynomial or seasonal trend. A band-limited spectral estimation procedure is proposed for estimating the coefficients of such a process and the asymptotic properties of the estimators investigated. The asymptotic theory is illustrated with reference to simulated and real data. A preliminary investigation of the use of Akaike's AIC criterion and this procedure to determine the number of roots of unit modulus (in the case where this is unknown) is also carried out by means of simulation.

The proposed band-limited spectral estimation procedure can also be used to take account of other possible effects met in practice. These include, for example, the band-limited response of a recording device or trend-contaminated low-frequency components.

It is shown how to estimate centroids and other measures of location of an unknown embedded subset of a bounded specimen by random sectioning.

This paper investigates the probabilistic behaviour of the eigenvalue of the empirical transition matrix of a Markov chain which is of largest modulus other than 1, loosely called the second-largest eigenvalue. A central limit theorem is obtained for nonmultiple eigenvalues of the empirical transition matrix. When the Markov chain is actually a sequence of independent observations the distribution of the second-largest eigenvalue is determined and a test for independence is developed. The independence case is considered in more detail when the Markov chain has only two states, and some applications are given.

It is assumed that customers at a service facility have appointments at times 0,1,2, … for which they may be unpunctual by random amounts or may never arrive at all. A weak convergence theorem is proved for the process which counts the number of arrivals. This makes it possible to carry over the results of Iglehart and Whitt (1970a) to obtain heavy traffic functional limit theorems for queues with arrivals by appointment.

Consider a number of events in a probability space. Let X be a random variable that is the number of events that occur. Given some of the moments of the distribution of X, it is possible to obtain bounds on the probability that at least one event occurs. The best possible bounds are given here. If there are many equiprobable events that are d- wise independent, and d is even, then the probability that at least one event happens is at least 1 — O(µ–d/2), where μ = E(X).

Some consequences of a modified repair system for Phillips' (1981a, b) model for a two-component system are discussed. In the original model, both components are repaired whenever a revealed fault occurs; in the modified model only faulty components are repaired. Specifically (i) the distribution of time from the initial state up to discovery of an unrevealed fault, (ii) the expected proportion of time during which there exists an unrepaired fault, and (iii) the distribution of number of revealed faults up to and including the one which leads to a discovery of an unrevealed fault, are obtained. The theory is illustrated by examples, based on specific distributions for the times between repairs and occurrences of the two types of faults. A characterization of the exponential distribution is indicated.

Well-known inequalities for the spectral gap of a discrete-time Markov chain, such as Poincaré's and Cheeger's inequalities, do not perform well if the transition graph of the Markov chain is strongly connected. For example in the case of nearest-neighbour random walk on the n-dimensional cube Poincaré's and Cheeger's inequalities are off by a factor n. Using a coupling technique and a contraction principle lower bounds on the spectral gap can be derived. Finally, we show that using the contraction principle yields a sharp estimate for nearest-neighbour random walk on the n-dimensional cube.

An expression for the first-passage density of Brownian motion to a curved boundary due to Daniels and Lerche is shown to give the same result as a different form due to the author. The equivalence is extended to continuous Gaussian Markov processes.

Let Z(t) ··· (Z1(t), …, Zk (t)) be an indecomposable critical k-type age-dependent branching process with generating function F(s, t). Denote the right and left eigenvalues of the mean matrix M by u and v respectively and suppose μ is the vector of mean lifetimes, i.e. Mu = u, vM = v.

It is shown that, under second moment assumptions, uniformly for s ∈ ([0, 1]k of the form s = 1 – cu, c a constant. Here vμ is the componentwise product of the vectors and Q[u] is a constant.

This result is then used to give a new proof of the exponential limit law.

Vinogradov (1973) used the Laplace transform to characterize the IFR class of life distributions and later Block and Savits (1980) extended the characterization to the main reliability classes. Here we use the same transform again to characterize the continuous time renewal equation and some properties of its solution.

Let (Xt) and (Yt) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (Xt) and (Yt) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.