Two recent papers by Petruccelli and Woolford (1984) and Chan et al. (1985) showed that the key element governing ergodicity of a threshold AR(1) model is the joint behavior of the two linear AR(1) pieces falling in the two boundary threshold regimes. They used essentially the necessary and sufficient conditions for ergodicity of a general Markov chain of Tweedie (1974), (1975) in a rather clever manner. However, it is difficult to extend the results to the more general threshold ARMA models. Besides, irreducibility is also required to apply Tweedie's results. In this paper, instead of pursuing the ideas in Tweedie's results, we shall develop a criterion similar in spirit to the technique used by Beneš (1967) in the context of continuous-time Markov chains. Consequently, we derive a necessary and sufficient condition for existence of a strictly stationary solution of a general non-linear ARMA model to be introduced in Section 2 of this paper. This condition is then applied to the threshold ARMA(1, q) model to yield a sufficient condition for strict stationarity which is identical to the condition given by Petruccelli and Woolford (1984) for the threshold AR(1). Hence, the conjecture that the moving average component does not affect stationarity is partially verified. Furthermore, under an additional irreducibility assumption, ergodicity of a non-linear ARMA model is established. The paper then concludes with a necessary condition for stationarity of the threshold ARMA(1, q) model.

In this paper, we study the new better than used in expectation (NBUE) and new worse than used in expectation (NWUE) properties of Markov renewal processes. We show that a Markov renewal process belongs to a more general class of stochastic processes encountered in reliability or maintenance applications. We present sufficient conditions such that the first-passage times of these processes are new better than used in expectation. The results are applied to the study of shock and repair models, random repair time processes, inventory, and queueing models.

The Hammersley–Clifford theorem gives the form that the joint probability density (or mass) function of a Markov random field must take. Its exponent must be a sum of functions of variables, where each function in the summand involves only those variables whose sites form a clique. From a statistical modeling point of view, it is important to establish the converse result, namely, to give the conditional probability specifications that yield a Markov random field. Besag (1974) addressed this question by developing a one-parameter exponential family of conditional probability models. In this article, we develop new models for Markov random fields by establishing sufficient conditions for the conditional probability specifications to yield a Markov random field.

This paper considers the undershoot of a general continuous-time risk process with dependent increments under a certain initial level. The increments are given by the locations and amounts of claims which are described by a stationary marked point process. Under a certain balance condition, it is shown that the distribution of the undershoot depends only on the mark distribution and on the intensity of the underlying point process, but not on the form of its distribution. In this way an insensitivity property is extended which has been proved in Björk and Grandell [3] for the ruin probability, i.e. for the probability that after a finite time interval the initial level will be crossed from above.

We consider the class of bandit problems in which each of the n ≧ 2 independent arms generates rewards according to one of the same two reward distributions, and discounting is geometric over an infinite horizon. We show that the dynamic allocation index of Gittins and Jones (1974) in this context is strictly increasing in the probability that an arm is the better of the two distributions. It follows as an immediate consequence that myopic strategies are the uniquely optimal strategies in this class of bandit problems, regardless of the value of the discount parameter or the shape of the reward distributions. Some implications of this result for bandits with Bernoulli reward distributions are given.

We study a class of simulated annealing algorithms for global minimization of a continuous function defined on a subset of We consider the case where the selection Markov kernel is absolutely continuous and has a density which is uniformly bounded away from 0. This class includes certain simulated annealing algorithms recently introduced by various authors. We show that, under mild conditions, the sequence of states generated by these algorithms converges in probability to the global minimum of the function. Unlike most previous studies where the cooling schedule is deterministic, our cooling schedule is allowed to be adaptive. We also address the issue of almost sure convergence versus convergence in probability.

A generalization of the block replacement policy (BRP) is proposed and analysed. Under such a policy, an operating system is preventively replaced at times kT (k = 1, 2, 3, ···), independently of its failure history. At failure an operating system is either replaced by a new or a used one or minimally repaired or remains inactive until the next planned replacement. The cost of the ith minimal repair of the new subsystem at age y depends on the random part C(y) and the deterministic part ci(y). The mathematical model is defined and general analytical results are obtained.

The asymptotic behaviour of sequences of Markov processes whose finite distributions depend upon the sample paths ω of a positive recurrent Markov chain ξ is studied. The existence of such sequences depends upon the existence of a unique class of directed weighted circuits having a probabilistic interpretation in terms of the directed circuits occurring along the sample paths of ξ. An application to multiple Markov chains is given.

Limit Statements obtainable by the key renewal theorem are of the form EXt = v(t) + o(1), as t →∞. We show how to delineate the limit function v for processes X associated with crudely regenerative phenomena. Included are refinements of classical limit theorems for Markov and regenerative processes, limits of sums of stationary random variables, and limits for integrals and derivatives of EXt.

In response to the computational complexity of the dynamic programming/backwards induction approach to the development of optimal policies for semi-Markov decision processes, we propose a class of heuristics resulting from an inductive process which proceeds forwards in time. These heuristics always choose actions in such a way as to minimize some measure of the current cost rate. We describe a procedure for calculating such cost rate heuristics. The quality of the performance of such policies is related to the speed of evolution (in a cost sense) of the process. A simple model of preventive maintenance is described in detail. Cost rate heuristics for this problem are calculated and assessed computationally.

The work is concerned with the first-order linear autoregressive process which has a rectangular stationary marginal distribution. A derivation is given of the result that the time-reversed version is deterministic, with a first-order recursion function of the type used in multiplicative congruential random number generators, scaled to the unit interval. The uniformly distributed sequence generated is chaotic, giving an instance of a chaotic process which when reversed has a linear causal and non-chaotic structure. An mk-valued discrete process is then introduced which resembles a first-order linear autoregressive model and uses k-adic arithmetic. It is a particular form of moving-average process, and when reversed approximates in m a non-linear discrete-valued process which has the congruential generator function as its deterministic part, plus a discrete-valued noise component. The process is illustrated by scatter plots of adjacent values, time series plots and directed scatter plots (phase diagrams). The behaviour very much depends on the adic number, with k = 2 being very distinctly non-linear and k = 10 being virtually indistinguishable from independence.

By studying the minimum of moving maxima, that is the maxima taken over a sliding window of length k in an i.i.d. sequence, we obtain new results on the reliability of consecutive k-out-of-n systems. In particular, we give the reliability asymptotically with both k and n varying. The underlying method of our approach is to analyze the singularities of a generating function.

In this paper we first prove an arrangement-decreasing property of partial sums of independent random variables when they are partially ordered through the likelihood ratio ordering. We then apply a similar argument to obtain a stochastic ordering of random processes via a comparison of their parameter functions, with special applications to Poisson and Wiener processes. Finally, in Section 4 we present some applications in reliability theory, queueing, and first-passage problems.

We apply the general theory of stochastic integration to identify a martingale associated with a Lévy process modified by the addition of a secondary process of bounded variation on every finite interval. This martingale can be applied to queues and related stochastic storage models driven by a Lévy process. For example, we have applied this martingale to derive the (non-product-form) steady-state distribution of a two-node tandem storage network with Lévy input and deterministic linear fluid flow out of the nodes.

It is well known that most commonly used discrete distributions fail to belong to the domain of maximal attraction for any extreme value distribution. Despite this negative finding, C. W. Anderson showed that for a class of discrete distributions including the negative binomial class, it is possible to asymptotically bound the distribution of the maximum. In this paper we extend Anderson's result to discrete-valued processes satisfying the usual mixing conditions for extreme value results for dependent stationary processes. We apply our result to obtain bounds for the distribution of the maximum based on negative binomial autoregressive processes introduced by E. McKenzie and Al-Osh and Alzaid. A simulation study illustrates the bounds for small sample sizes.

We present some monotonicity and convexity properties for the sequence of partial sums associated with a sequence of non-negative independent identically distributed random variables. These results are applied to a system of parallel queues with Bernoulli routing, and are useful in establishing a performance comparison between two scheduling strategies in multiprocessor systems.

The generating functions for the serial covariances for number in system in the stationary GI/M/1 bulk arrival queue with fixed bulk sizes, and the GI/Em/1 queue, are derived. Expressions for the infinite sum of the serial correlation coefficients are also presented, as well as the first serial correlation coefficient in the case of the bulk arrival queue. Several numerical examples are considered.

In an ergodic network of K M/M/1 queues in series we consider the rare event that, as N increases, the total population in the network exceeds N during a busy period. By utilizing the contraction principle of large deviation theory, an action functional is obtained for this exit problem. The ensuing minimization is carried out for K = 2 and an indication is given for arbitrary K. It is shown that, asymptotically and for unequal service rates, the ‘most likely' path for this rare event is one where the arrival rate has been interchanged with the smallest service rate. The problem has been posed in Parekh and Walrand [7] in connection with importance sampling simulation methods for queueing networks. Its solution has previously been obtained only heuristically.

By using the alternating projection theorem of J. von Neumann, we obtain explicit formulae for the best linear interpolator and interpolation error of missing values of a stationary process. These are expressed in terms of multistep predictors and autoregressive parameters of the process. The key idea is to approximate the future by a finite-dimensional space.

We study some distribution properties of a random sum of i.i.d. non-negative random variables, where the number of terms is geometrically distributed and not independent of the summands. The results are applied to the system failure time of a one-unit system with a single spare and repair facility. In such a system when the operating unit fails it is immediately replaced by the spare and sent to the repair facility. The system continues operating until the first time when the failed unit has not yet been repaired by the failure of the operating unit. Certain ageing properties such as NBU, NWU, NBUE, NWUE, HNBUE, HNWUE, L+ and L– are shown to be inheritable from the working time of the operating unit to the system lifetime.