The problem of optimal stopping in a Markov chain when there is imperfect state information is formulated as a partially observable Markov decision process. Properties of the optimal value function are developed. It is shown that under mild conditions the optimal policy is well structured. An efficient algorithm, which uses the structural information in the computation of the optimal policy, is presented.

Several recent papers have shown that for the M/G/1/n queue with equal arrival and service rates, the expected number of lost customers per busy cycle is equal to 1 for every n ≥ 0. We present an elementary proof based on Wald's equation and, for GI/G/1/n, obtain conditions for this quantity to be either less than or greater than 1 for every n ≥ 0. In addition, we extend this result to batch arrivals, where, for average batch size β, the same quantity is either less than or greater than β. We then extend these results to general ways that customers may be lost, to an arbitrary order of service that allows service interruption, and finally to reneging.

Let (X, J) denote a Markov-modulated Brownian motion (MMBM) and denote its supremum process by S. For some a > 0, let σ(a) denote the time when the reflected process Y := S - X first surpasses the level a. Furthermore, let σ−(a) denote the last time before σ(a) when X attains its current supremum. In this paper we shall derive the joint distribution of Sσ(a), σ−(a), and σ(a), where the latter two will be given in terms of their Laplace transforms. We also provide some remarks on scale matrices for MMBMs with strictly positive variation parameters. This extends recent results for spectrally negative Lévy processes to MMBMs. Due to well-known fluid embedding and state-dependent killing techniques, the analysis applies to Markov additive processes with phase-type jumps as well. The result is of interest to applications such as the dividend problem in insurance mathematics and the buffer overflow problem in queueing theory. Examples will be given for the former.

In this paper, we investigate the asymptotic behaviour of controlled branching processes with random control functions. In a critical case, we establish sufficient conditions for both their almost-sure extinction and for their nonextinction with a positive probability. For some suitably chosen norming constants, we also determine different kinds of limiting behaviour for this class of processes.

We provide explicit expressions for the constants involved in the characterisation of ergodicity of subgeometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The results are fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our results accommodate also some classes of inhomogeneous chains.

When is the product of the d.f.'s H1(·), ···, Hm(·) attracted to an extreme value law φ(x)? We associate with each Hi(·) its A-function Hi(x) is attracted to φ(x) if each Hi(x) is in the domain of attraction of φ(x) and Ai(z) ~ Aj(z), 1 ≦ i, j ≦ m. Equivalence of A-functions determines an equivalence relation which partitions the domain of attraction of φ(x)into one or more convex sets. These sets fail to be closed under passages to the limit (complete convergence).

Quasi-stationary distributions, as discussed in Darroch and Seneta (1965), have been used in biology to describe the steady state behaviour of population models which, while eventually certain to become extinct, nevertheless maintain an apparent stochastic equilibrium for long periods. These distributions have some drawbacks: they need not exist, nor be unique, and their calculation can present problems. In this paper, we give biologically plausible conditions under which the quasi-stationary distribution is unique, and can be closely approximated by distributions that are simple to compute.

In this paper we consider improvements in the rate of approximation for the distribution of sums of independent Bernoulli random variables via convolutions of Poisson measures with signed measures of specific type. As a special case, the distribution of the number of records in an i.i.d. sequence of length n is investigated. For this particular example, it is shown that the usual rate of Poisson approximation of O(1/log n) can be lowered to O(1/n2). The general case is discussed in terms of operator semigroups.

Under weak conditions, this paper provides a best lower and a best upper bounding fractional linear generating function for any probability generating function when they have the same mean. These bounds can be used to obtain bounds for the expectation and the percentiles of the extinction-time distribution of a Galton-Watson branching process and other parameters of interest. For the special case of the four points probability generating function, the best bounds obtained are better than the bounds derived by Agresti (1974).

For finite Markov chains, the concepts of ergodicity and strong ergodicity are equivalent, but this is not necessarily the case when the state space is infinite. In this note we give some new characterizations of strong ergodicity. These lead to simple necessary or sufficient criteria for strong ergodicity, which readily enable us to classify a number of examples.

We will consider a branching process with m > 1 distinguishable particle types as follows. At time 0, one newly born cell of type i is born (i = 1, 2, ···, m). Cell type i lives a random lifetime with continuous distribution function Gi(t), Gi(0+) = 0. At the end of its life, cell i is replaced by j1 new cells of type 1, j2 new cells of type 2, ···, jm new cells of type m with probability , and we define the generating functions for i = 1,···,m, where and . Each new daughter cell proceeds independently of the state of the system, with each cell type j governed by Gj(t) and hj(s).

In this paper we use the exit time theory for Lévy processes to derive new closed-form results for the busy period distribution of finite-capacity fluid M/G/1 queues. Based on this result, we then obtain the busy period distribution for finite-capacity queues with on–off inputs when the off times are exponentially distributed.

Let Xi, i = 1, 2, 3,… be a sequence of independent and identically distributed random variables with law ℓ(X) and write. if EX= 0 and EX2=σ2< ∞, the law of the iterated logarithm (Hartman and Wintner [1]) tells us that

Limit theorems for the Galton–Watson process with immigration (BPI), where immigration is not permitted when the process is in state 0 (so that this state is absorbing), have been studied for the subcritical and supercritical cases by Seneta and Tavaré (1983). It is pointed out here that, apart from a change of context, the corresponding theorem in the critical case has been obtained by Vatutin (1977). Extensions which follow from a more general form of initial distribution are sketched, including a new form of limit result (7).

Smoluchowski's classical analysis of the temporal fluctuation, under diffusion equilibrium, of the number of particles in a fixed region R of space is generalised to a set of disjoint regions; specifically, the single Smoluchowski region is divided into a finite number of non-intersecting subregions. The generalisation allows a more rigorous test of some of the consequences of the Einstein-Smoluchowski theory of Brownian motion to be carried out, and at the same time enables the Avogadro constant to be estimated with greater precision than is possible with the single region. In particular, the reversibility paradox of Loschmidt and the recurrence paradox of Zermelo are reexamined from the point of view of the fluctuation of configurations (a configuration being defined as the set of occupation numbers for the various subregions) rather than that of total concentration for the single region.

We consider the problem of maximizing the probability of choosing the largest from a sequence of N observations when N is a bounded random variable. The present paper gives a necessary and sufficient condition, based on the distribution of N, for the optimal stopping rule to have a particularly simple form: what Rasmussen and Robbins (1975) call an s(r) rule. A second result indicates that optimal stopping rules for this problem can, with one restriction, take virtually any form.

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

We extend some results of Hammersley and Welsh concerning first-passage percolation on the two-dimensional integer lattice. Our results include: (i) weak renewal theorems for the unrestricted reach processes; (ii) an L1-ergodic theorem for the unrestricted first-passage time from (0, 0) to the line X = n; and (iii) weakening of the boundedness restrictions on the underlying distribution in Hammersley and Welsh's weak renewal theorems for the cylinder reach processes.

To evaluate the benefits and risks associated with screening for disease, a model is developed to characterize the changes in incidence and survival distributions effected by a screening program. Screening is presumed to increase survival by resulting in diagnosis of disease at earlier stages. All disease states in the model are observable, thus facilitating application to empirical data. An example of such an application using data from a breast cancer detection project is given for the case of one screening for two-stage disease having Weibull-distributed diagnosis times.

A two-server loss system is considered with N classes of Poisson arrivals, where the service distribution function and server preferences are arrival-class dependent. The stationary state probabilities are derived and found to be independent of the form of the service distributions.