Likelihood ratios are used in computer simulation to estimate expectations with respect to one law from simulation of another. This importance sampling technique can be implemented with either the likelihood ratio at the end of the simulated time horizon or with a sequence of likelihood ratios at intermediate times. Since a likelihood ratio process is a martingale, the intermediate values are conditional expectations of the final value and their use introduces no bias.

We provide conditions under which using conditional expectations in this way brings guaranteed variance reduction. We use stochastic orderings to get positive dependence between a process and its likelihood ratio, from which variance reduction follows. Our analysis supports the following rough statement: for increasing functionals of associated processes with monotone likelihood ratio, conditioning helps. Examples are drawn from recursively defined processes, Markov chains in discrete and continuous time, and processes with Poisson input.

The two-point Markov chain boundary-value problem discussed in this paper is a finite-time version of the quasi-stationary behaviour of Markov chains. Specifically, for a Markov chain {Xt:t = 0, 1, ·· ·}, given the time interval (0, n), the interest is in describing the chain at some intermediate time point r conditional on knowing both the behaviour of the chain at the initial time point 0 and that over the interval (0, n) it has avoided some subset B of the state space. The paper considers both ‘real time' estimates for r = n (i.e. the chain has avoided B since 0), and a posteriori estimates for r < n with at least partial knowledge of the behaviour of Xn. Algorithms to evaluate the distribution of Xr can be as small as O(n3) (and, for practical purposes, even O(n2 log n)). The estimates may be stochastically ordered, and the process (and hence, the estimates) may be spatially homogeneous in a certain sense. Maximum likelihood estimates of the sample path are furnished, but by example we note that these ML paths may differ markedly from the path consisting of the expected or average states. The scope for two-point boundary-value problems to have solutions in a Markovian setting is noted.

Several examples are given, together with a discussion and examples of the analogous problem in continuous time. These examples include the basic M/G/k queue and variants that include a finite waiting room, reneging, balking, and Bernoulli feedback, a pure birth process and the Yule process. The queueing examples include Larson's (1990) ‘queue inference engine'.

We define a class of two-dimensional Markov random graphs with I, V, T and Y-shaped nodes (vertices). These are termed polygonal models. The construction extends our earlier work [1]– [5]. Most of the paper is concerned with consistent polygonal models which are both stationary and isotropic and which admit an alternative description in terms of the trajectories in space and time of a one-dimensional particle system with motion, birth, death and branching. Examples of computer simulations based on this description are given.

A manpower system of constant size, controlled by recruitment, is described by a partially stochastic model in which there are fixed promotion rates, no demotions and stochastic wastage. The geometric-probabilistic relationship is examined for the attainability after one step and after two steps.

Given a parametric family of regenerative processes on a common probability space, we investigate when the derivatives (with respect to the parameter) are regenerative. We primarily consider sequences satisfying explicit, Lipschitz recursions, such as the waiting times in many queueing systems, and show that derivatives regenerate together with the original sequence under reasonable monotonicity or continuity assumptions. The inputs to our recursions are i.i.d. or, more generally, governed by a Harris-ergodic Markov chain. For i.i.d. input we identify explicit regeneration points; otherwise, we use coupling arguments. We give conditions for the expected steady-state derivative to be the derivative of the steady-state mean of the original sequence. Under these conditions, the derivative of the steady-state mean has a cycle-formula representation.

While the convex hull of n d-dimensional balls is not a polytope, it does have an underlying combinatorial structure similar to that of a polytope. In the worst case, its combinatorial complexity can be of order Ω(n[d/2]). The thrust of this work is to show that its complexity is typically much smaller, and that it can therefore be constructed more quickly on average than in the worst case. To this end, four models of the random d-ball are developed, and the expected combinatorial complexity of the convex hull of n independent random d-balls is investigated for each. As n grows without bound, these expectations are O(1), O(n(d–1)/(d+4)), O(1) (for d = 2 only), and O(n(d–1)/(d+3)).

In this paper, optimal stopping problems for semi-Markov processes are studied in a fairly general setting. In such a process transitions are made from state to state in accordance with a Markov chain, but the amount of time spent in each state is random. The times spent in each state follow a general renewal process. They may depend on the present state as well as on the state into which the next transition is made.

Our goal is to maximize the expected net return, which is given as a function of the state at time t minus some cost function. Discounting may or may not be considered. The main theorems (Theorems 3.5 and 3.11) are expressions for the optimal stopping time in the undiscounted and discounted case. These theorems generalize results of Zuckerman [16] and Boshuizen and Gouweleeuw [3]. Applications are given in various special cases.

The results developed in this paper can also be applied to semi-Markov shock models, as considered in Taylor [13], Feldman [6] and Zuckerman [15].

For two-dimensional spatial data, a spatial unilateral autoregressive moving average (ARMA) model of first order is defined and its properties studied. The spatial correlation properties for these models are explicitly obtained, as well as simple conditions for stationarity and conditional expectation (interpolation) properties of the model. The multiplicative or linear-by-linear first-order spatial models are seen to be a special case which have proved to be of practical use in modeling of two-dimensional spatial lattice data, and hence the more general models should prove to be useful in applications. These unilateral models possess a convenient computational form for the exact likelihood function, which gives proper treatment to the border cell values in the lattice that have a substantial effect in estimation of parameters. Some simulation results to examine properties of the maximum likelihood estimator and a numerical example to illustrate the methods are briefly presented.

The accuracy of the Poisson approximation to the distribution of the numbers of large and small m-spacings, when n points are placed at random on the circle, was analysed using the Stein–Chen method in Barbour et al. (1992b). The Poisson approximation for m≧2 was found not to be as good as for 1-spacings. In this paper, rates of approximation of these distributions to suitable compound Poisson distributions are worked out, using the CP–Stein–Chen method and an appropriate coupling argument. The rates are better than for Poisson approximation for m≧2, and are of order O((log n)2/n) for large m-spacings and of order O(1/n) for small m-spacings, for any fixed m≧2, if the expected number of spacings is held constant as n → ∞.

A new approach to the problem of classification of (deflected) random walks in or Markovian models for queueing networks with identical customers is introduced. It is based on the analysis of the intrinsic dynamical system associated with the random walk. Earlier results for small dimensions are presented from this novel point of view. We give proofs of new results for higher dimensions related to the existence of a continuous invariant measure for the underlying dynamical system. Two constants are shown to be important: the free energy M < 0 corresponds to ergodicity, the Lyapounov exponent L < 0 defines recurrence. General conjectures, examples, unsolved problems and surprising connections with ergodic theory, classical dynamical systems and their random perturbations are largely presented. A useful notion naturally arises, the so-called scaled random perturbation of a dynamical system.

Non-linear stochastic systems driven by white noise are analysed from the viewpoint of non-linear oscillation theory. Under various familiar hypotheses concerning dissipative and restorative dynamical forces, the existence and uniqueness, asymptotic growth, and oscillatory behavior of the solutions are demonstrated.

Let , be i.i.d. random closed sets in . Limit theorems for their normalized convex hulls conv () are proved. The limiting distributions correspond to C-stable random sets. The random closed set A is called C-stable if, for any , the sets anA and conv ( coincide in distribution for certain positive an, compact Kn, and independent copies A1, …, An of A. The distributions of C-stable sets are characterized via corresponding containment functionals.

When a subsystem goes into an infrequent state, how does the remainder of the system behave? We show how to calculate the relevant distributions using the notions of reversed time for Markov processes and large deviations. For ease of exposition, most of the work deals with a specific queueing model due to Flatto, Hahn, and Wright. However, we show how the theorems may be applied to much more general jump-Markov systems.

We also show how the tools of time-reversal and large deviations complement each other to yield general theorems. We show that the way a constant coefficient process approaches a rare event is roughly by following the path of another constant coefficient process. We also obtain some properties, including apriori bounds, for the change of measure associated with some large deviations functionals; these are useful for accelerating simulations.

In this paper we study the following general class of concurrent processing systems. There are several different classes of processors (servers) and many identical processors within each class. There is also a continuous random flow of jobs, arriving for processing at the system. Each job needs to engage concurrently several processors from various classes in order to be processed. After acquiring the needed processors the job begins to be executed. Processing is done non-preemptively, lasts for a random amount of time, and then all the processors are released simultaneously. Each job is specified by its arrival time, its processing time, and the list of processors that it needs to access simultaneously. The random flow (sequence) of jobs has a stationary and ergodic structure. There are several possible policies for scheduling the jobs on the processors for execution; it is up to the system designer to choose the scheduling policy to achieve certain objectives.

We focus on the effect that the choice of scheduling policy has on the asymptotic behavior of the system at large times and especially on its stability, under general stationary and ergocic input flows.

A stochastic process, called reallocatable GSMP (RGSMP for short), is introduced in order to study insensitivity of its stationary distribution. RGSMP extends GSMP with interruptions, and is applicable to a wide range of queues, from the standard models such as BCMP and Kelly's network queues to new ones such as their modifications with interruptions and Serfozo's (1989) non-product form network queues, and can be used to study their insensitivity in a unified way. We prove that RGSMP supplemented by the remaining lifetimes is product-form decomposable, i.e. its stationary distribution splits into independent components if and only if a version of the local balance equations hold, which implies insensitivity of the RGSMP scheme in a certain extended sense. Various examples of insensitive queues are given, which include new results. Our proofs are based on the characterization of a stationary distribution for SCJP (self-clocking jump process) of Miyazawa (1991).

We consider a controlled i.i.d. process, where several i.i.d. sources are sampled sequentially. At each time instant, a controller determines from which source to obtain the next sample. Any causal sampling policy, possibly history-dependent, may be employed. The purpose is to characterize the extremal large deviations of the sample mean, namely to obtain asymptotic rate bounds (similar to and extending Cramér's theorem) which hold uniformly over all sampling policies. Lower and upper bounds are obtained, and it is shown that in many (but not all) cases stationary sampling policies are sufficient to obtain the extremal large deviations rates. These results are applied to a hypothesis testing problem, where data samples may be sequentially chosen from several i.i.d. sources (representing different types of experiments). The analysis provides asymptotic estimates for the error probabilities, corresponding both to optimal and to worst-case sampling policies.

A problem of optimal stopping of the discrete-time Markov process by two decision-makers (Player 1 and Player 2) in a competitive situation is considered. The zero-sum game structure is adopted. The gain function depends on states chosen by both decision-makers. When both players want to accept the realization of the Markov process at the same moment, the priority is given to Player 1. The construction of the value function and the optimal strategies for the players are given. The Markov chain case is considered in detail. An example related to the generalized secretary problem is solved.

Stochastic majorization is a tool that has been used in many areas of probability and statistics (such as multivariate statistical analysis, queueing theory and reliability theory) in order to obtain useful bounds and inequalities. In this paper we study the relations among several notions of stochastic majorization and stochastic convexity and obtain sufficient (and sometimes necessary) conditions which imply some of these notions. Extensions and generalizations of several results in the literature are obtained. Some examples and applications regarding stochastic comparisons of order statistics are also presented in order to illustrate the results of the paper.

A single-server retrial queue consists of a primary queue, an orbit and a single server. Assume the primary queue capacity is 1 and the orbit capacity is infinite. Customers can arrive at the primary queue either from outside the system or from the orbit. If the server is busy, the arriving customer joins the orbit and conducts a retrial later. Otherwise, he receives service and leaves the system.

We investigate the stability condition for a single-server retrial queue. Let λ be the arrival rate and 1/μ be the mean service time. It has been proved that λ/μ < 1 is a sufficient stability condition for the M/G/1/1 retrial queue with exponential retrial times. We give a counterexample to show that this stability condition is not valid for general single-server retrial queues. Next we show that λ /μ < 1 is a sufficient stability condition for the stability of a single-server retrial queue when the interarrival times and retrial times are finite mixtures of Erlangs.

The paper deals with asymptotic stationarity of the process where is a vector in with non-negative coordinates, is an -valued process, S is a separable metric space and all operations in are meant in the coordinate-wise sense. It is shown that a type of asymptotic stationarity of (X, Y), together with some conditions, implies the same type of asymptotic stationarity of (w, X, Y). This result is applied to analyze asymptotic stationarity of multichannel queues. It may also be used to analyze asymptotic stationarity of series of multichannel queues.