We obtain an approximation to the mean time to extinction and to the quasi-stationary distribution for the standard S–I–S epidemic model introduced by Weiss and Dishon (1971). These results are a combination and extension of the results of Norden (1982) for the stochastic logistic model, Oppenheim et al. (1977) for a model on chemical reactions, Cavender (1978) for the birth-and-death processes and Bartholomew (1976) for social diffusion processes.

Consider a population of distinct species Sj, j∈J, members of which are selected at different time points T1, T2,· ··, one at each time. Assume linear costs per unit of time and that a reward is earned at each discovery epoch of a new species. We treat the problem of finding a selection rule which maximizes the expected payoff. As the times between successive selections are supposed to be continuous random variables, we are dealing with a continuous-time optimal stopping problem which is the natural generalization of the one Rasmussen and Starr (1979) have investigated; namely, the corresponding problem with fixed times between successive selections. However, in contrast to their discrete-time setting the derivation of an optimal strategy appears to be much harder in our model as generally we are no longer in the monotone case.

This note gives a general point process formulation for this problem, leading in particular to an equivalent stopping problem via stochastic intensities which is easier to handle. Then we present a formal derivation of the optimal stopping time under the stronger assumption of i.i.d. (X1, A1) (X2, A2), · ·· where Xn gives the label (j for Sj) of the species selected at Tn and An denotes the time between the nth and (n – 1)th selection, i.e. An = Tn – Tn–1. In the case where even Xn and An are independent and An has an IFR (increasing failure rate) distribution, an explicit solution for the optimal strategy is derived as a simple consequence.

Special symmetry conditions on the transition p.d.f. of one-dimensional time-homogeneous diffusion processes with natural boundaries are investigated and exploited to derive closed-form results concerning the transition p.d.f.'s in the presence of absorbing and reflecting boundaries and the first-passage-time p.d.f. through time-dependent boundaries.

We consider a class of secretary problems in which the order of arrival of candidates is no longer uniformly distributed. By a suitable embedding in a time-transformed extremal process it is shown that the asymptotic winning probability is again 1/e as in the classical situation. Extensions of the problem to more than one choice are also considered.

A non-degenerate distribution function F is called maximum stable with random sample size if there exist positive integer random variables Nn, n = 1, 2, ···, with P(Nn = 1) less than 1 and tending to 1 as n → ∞ and such that F and the distribution function of the maximum value of Nn independent observations from F (and independent of Nn) are of the same type for every index n. By proving the converse of an earlier result of the author, it is shown that the set of all maximum stable distribution functions with random sample size consists of all distribution functions F satisfying where c2, c3, · ·· are arbitrary non-negative constants with 0 < c2 + c3 + · ·· <∞, and all distribution functions G and H defined by F(x)= G(c + exp(x)) and F(x) = H(c – exp(–x)), –∞ < x <∞, where c is an arbitrary real constant.

Sojourn times of Markov processes in subsets of the finite state space are considered. We give a closed form of the distribution of the nth sojourn time in a given subset of states. The asymptotic behaviour of this distribution when time goes to infinity is analyzed, in the discrete time and the continuous-time cases. We consider the usually pseudo-aggregated Markov process canonically constructed from the previous one by collapsing the states of each subset of a given partition. The relation between limits of moments of the sojourn time distributions in the original Markov process and the moments of the corresponding holding times of the pseudo-aggregated one is also studied.

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).

This paper is a generalization to Markov chains of the work of Shepp [6] in the i.i.d case. Shepp studies the limiting values of the averages Tn = (Sn+ f(n) – Sn)/f(n) where Sn = X0 + X1+ · ·· + Xn, X0 = 0, n = 1, 2, ···, is a sum of mutually independent and identically distributed random variables. The function f takes positive integer values and non-decreasingly tends to infinity. Here we take a class of functions f in central position f(n) = [c log n], c > 0, n = 1, 2, ···. There are many refinements of the function f in the i.i.d case [1], [2]. Here we consider the more general case where X1, · ··, Xn is an irreducible and recurrent Markov chain. The state space of the chain is either compact or countable.

The aims of this paper are twofold. The first is to study the rate of convergence of the sequence of vectors of expectations, variances and covariances in NHMS. It is proved that under certain conditions easily met in practice the rate is geometric. The second is to study the rate of convergence of the same sequence when the NHMS is under cyclic behaviour and it is also proved that the rate of convergence of the d subsequences (d being the period of the cycle) in which the sequence splits is geometric.

An adaptive-type exponential smoothing, motivated by an insurance tariff problem, is treated. We consider the process Zn = ß(Zn –1)Xn +(1 – ß (Zn–1))Zn–1, where Xn are i.i.d. taking values in the interval [0, M], M ≦ ∞ and ß is a monotonically increasing function [0, M] → [c, d], 0 < c < d < 1.

Together with (Zn), we consider the ordinary exponential smoothing Yn = αXn + (1 – α)Yn –1 where α is a constant, 0 < α < 1. We show that (Yn) and (Zn) are geometrically ergodic Markov chains (in the case of finite interval we even have uniform ergodicity) and that EYn, EZn converge to limits EY, EZ, respectively, with a geometric convergence rate. Moreover, we show that Ez is strictly less than EY = EXn.

We consider an additive shock process where shocks occur according to a Poisson point process and they are accumulated in an appropriate way to the damage. It is shown that suitably normalized shock processes converge weakly to a process which is represented as a sum of a stable process and a deterministic process.

Consider a sequence of Bernoulli trials between players A and B in which player A wins each trial with probability p∈ [0, 1]. For positive integers n and k with k ≦ n, an (n, k) contest is one in which the first player to win at least n trials and to be ahead of his opponent by at least k trials wins the contest. The (n, 1) contest is the Banach match problem and the (n, n) contest is the gambler's ruin problem. Many real contests (such as the World Series in baseball and the tennis game) have an (n, 1) or an (n, 2) format. The (n, k) contest is formulated in terms of the first-exit time of the graph of a random walk from a certain region of the state-time space. Explicit results are obtained for the probability that player A wins an (n, k) contest and the expected number of trials in an (n, k) contest. Comparisons of (n, k) contests are made in terms of the probability that the stronger player wins and the expected number of trials.

Interchange arguments are applied to establish the optimality of priority list policies in three problems. First, we prove that in a multiclass tandem of two ·/M/1 queues it is always optimal in the second node to serve according to the cµ rule. The result holds more generally if the first node is replaced by a multiclass network consisting of ·/M/1 queues with Bernoulli routing. Next, for scheduling a single server in a multiclass node with feedback, a simplified proof of Klimov's result is given. From it follows the optimality of the index rule among idling policies for general service time distributions, and among pre-emptive policies when the service time distributions are exponential. Lastly, we consider the problem of minimizing the blocking in a communication link with lossy channels and exponential holding times.

Consider a device that deteriorates in time according to an increasing Markov process so that it fails as soon as a critical threshold is exceeded. NBUE and NWUE properties of the lifetime of the device are identified to extend the existing literature on the PFr, IFR, IFRA, and NBU cases. In particular, it is shown that NBUE and NWUE characterizations can be made through a monotonicity property on the potential operator of the Markov process.

We consider a storage/production process where the production rate is state dependent, the demand arrival is a renewal process, and the sizes of the demands are i.i.d exponentially distributed random variables. The resulting content process is non-Markovian but regenerative. We construct a dual Markovian dam process with drift, jump rate and jump sizes that are state dependent and use it to compute the limiting one-dimensional distribution of the content process.

Several stochastic processes in queueing theory are based upon compound renewal processes . For queues in light traffic, however, the summands {Xk}and the renewal counting process {N(t)} are typically dependent on each other. Making use of recent invariance principles for such situations, we present some weak and strong approximations for the GI/G/1 queues in light and heavy traffic. Some applications are discussed including convergence rate statements or Darling–Erdös-type extreme value theorems for the processes under consideration.

Equations are derived for the distribution of the busy period of the GI/G/2 queue. The equations are analyzed for the M/G/2 queue, assuming that the service times have a density which is an arbitrary linear combination, with respect to both the number of stages and the rate parameter, of Erlang densities. The coefficients may be negative. Special cases and examples are studied.

A population process {Zn} is defined, with offspring distribution depending on both population size and generation number. As is well known, in models where the offspring distribution depends only on the size, under most conditions extinction surely results. The model we now introduce would mean that if history has an influence on evolution of the population, it is possible to obtain models for which a probability as high as desired exists that the population size balances between two given values.

This paper is concerned with the problem of controlling a simple immigration–birth process, which represents a pest population, by the introduction of catastrophes which, when they occur, reduce the population size to zero. The optimality criterion is that of minimising the long-term average cost per unit time of the process. Firstly, an optimal policy is found within a restricted class of stationary policies, which introduce catastrophes if and only if the population size is greater than or equal to some critical value x. The optimality of this policy within the wider class of all stationary policies is then verified by applying the general results of Bather (1976).

In this paper a bisexual Moran model is introduced. The population consists of N pairs of individuals. At times t = 1, 2, ·· ·two individuals are born, who ‘choose their parents randomly' and independently of each other. Then one of the pairs is removed and replaced by the two individuals born at that instant.

The extinction probability of the descendants of a single pair and the number of ancestors of a whole generation are studied. A limit result for large population sizes has been derived by diffusion approximation methods.