We consider the problem of approximating the stationary distribution of a positive-recurrent Markov chain with infinite transition matrix P, by stationary distributions computed from (n × n) stochastic matrices formed by augmenting the entries of the (n × n) northwest corner truncations of P, as n →∞.

This paper arose from interest in assessing the quality of random number generators. The problem of testing randomness of a string of binary bits produced by such a generator gained importance with the wide use of public key cryptography and the need for secure encryption algorithms. All such algorithms are based on a generator of (pseudo) random numbers; the testing of such generators for randomness became crucial for the communications industry where digital signatures and key management are vital for information processing.

The concept of approximate entropy has been introduced in a series of papers by S. Pincus and co-authors. The corresponding statistic is designed to measure the degree of randomness of observed sequences. It is based on incremental contrasts of empirical entropies based on the frequencies of different patterns in the sequence. Sequences with large approximate entropy must have substantial fluctuation or irregularity. Alternatively, small values of this characteristic imply strong regularity, or lack of randomness, in a sequence. Pincus and Kalman (1997) evaluated approximate entropies for binary and decimal expansions of e, π, √2 and √3 with the surprising conclusion that the expansion of √3 demonstrated much less irregularity than that of π. Tractable small sample distributions are hardly available, and testing randomness is based, as a rule, on fairly long strings. Therefore, to have rigorous statistical tests of randomness based on this approximate entropy statistic, one needs the limiting distribution of this characteristic under the randomness assumption. Until now this distribution remained unknown and was thought to be difficult to obtain. To derive the limiting distribution of approximate entropy we modify its definition. It is shown that the approximate entropy as well as its modified version converges in distribution to a χ2-random variable. The P-values of approximate entropy test statistics for binary expansions of e, π and √3 are plotted. Although some of these values for √3 digits are small, they do not provide enough statistical significance against the randomness hypothesis.

In this paper we investigate sufficient conditions that ensure the optimality of threshold strategies for optimal stopping problems with finite or perpetual maturities. Our result is based on a local-time argument that enables us to give an alternative proof of the smooth-fit principle. Moreover, we present a class of optimal stopping problems for which the propagation of convexity fails.

The purpose of this note is to extend results on critical Galton-Watson branching processes, due to Feller and Lamperti: if the number of individuals at time zero tends to infinity, the process converges, with two different normalizations, to certain diffusion processes in the functional form, discussed in, e.g., Billingsley.

In this paper we show that the problem of existence and uniqueness of stationary distributions for piecewise-deterministic Markov processes (PDPs) is equivalent to the same problem for the associated Markov chain, so long as some mild conditions on the parameters of the PDP are satisfied. Our main result is the construction of an invertible mapping from the set of stationary distributions for the PDP to the set of stationary distributions for the Markov chain. Some sufficient conditions for existence are presented and an application to capacity expansion is given.

Probability generation functions of waiting time distributions of runs and patterns have been used successfully in various areas of statistics and applied probability. In this paper, we provide a simple way to obtain the probability generating functions for waiting time distributions of compound patterns by using the finite Markov chain imbedding method. We also study the characters of waiting time distributions for compound patterns. A computer algorithm based on Markov chain imbedding technique has been developed for automatically computing the distribution, probability generating function, and mean of waiting time for a compound pattern.

We give a unified presentation of stability results for stochastic vector difference equations based on various choices of binary operations and , assuming that are stationary and ergodic. In the scalar case, under standard addition and multiplication, the key condition for stability is E[log |A0|] < 0. In the generalizations, the condition takes the form γ< 0, where γis the limit of a subadditive process associated with . Under this and mild additional conditions, the process has a unique finite stationary distribution to which it converges from all initial conditions.

The variants of standard matrix algebra we consider replace the operations + and × with (max, +), (max,×), (min, +), or (min,×). In each case, the appropriate stability condition parallels that for the standard recursions, involving certain subadditive limits. Since these limits are difficult to evaluate, we provide bounds, thus giving alternative, computable conditions for stability.

A class of multivariate exponential distributions is suggested which includes those presented by Marshall and Olkin, Downton, and Hawkes. It is based on a concept of hierarchical successive damage. Recursive expressions for the Laplace transforms and for first and second moments are derived in the bivariate case. Related multivariate geometric distributions are described.

This paper presents the theory of a multivariate birth-and-death process and its representation as a branching process. The bivariate linear birth-and-death process may be used as a model for various epidemic situations involving two types of infective. Various properties of the transient process are discussed and the distribution of epidemic size is investigated. For the case of a disease spread solely by carriers when the two types of infective are carriers and clinical infectives the large population version of a model proposed by Downton (1968) is further developed and shown under appropriate circumstances to closely approximate Downton's model.

Let be the scan statistic of window size r for a sequence of n bistate trials . The scan statistic Sn(r) has been successfully used in various fields of applied probability and statistics, and its distribution has been studied extensively in the literature. Currently, all existing formulae for the distribution of Sn(r) are rather complex, and they can only be numerically implemented when is a sequence of Bernoulli trials, the window size r is less than 20 and the length of the sequence n is not too large. Hence, these formulae have been limiting the practical applications of the scan statistic. In this article, we derive a simple and effective formula for the distribution of Sn(r) via the finite Markov chain embedding technique to overcome some of the limitations of the existing complex formulae. This new formula can be applied when is either a sequence of Bernoulli trials or a sequence of Markov dependent bistate trials. Selected numerical examples are given to illustrate our results.

In this paper, we study ruin probabilities in two generalized risk models. The effects of timing of payments and interest on the ruin probabilities in the models are considered. The rates of interest are assumed to have a dependent autoregressive structure. Generalized Lundberg inequalities for the ruin probabilities are derived by a renewal recursive technique. An illustrative application is given to the compound binomial risk process.

Corrections are made to formulations and proofs of some theorems about convolution equivalence closure for random sum distributions. These arise because of the falsity of a much used asymptotic equivalence lemma, and they impinge on the convolution equivalence closure theorem for general infinitely divisible laws.

We prove that the quasi-invariant measures associated to a Brownian motion with negative drift X form a one-parameter family. The minimal one is a probability measure inducing the transition density of a three-dimensional Bessel process, and it is shown that it is the density of the limit distribution limt→∞Px(X A | τ > t). It is also shown that the minimal quasi-invariant measure of infinite mass induces the density of the limit distribution ) which is the law of a Bessel process with drift.

A multitype branching process, the n-family community mating process, is introduced for the purpose of comparing extinction probabilities with those of bisexual Galton–Watson branching processes. Consideration of known properties of standard multitype branching processes leads to conditions which are both necessary and sufficient for extinction in a bisexual Galton–Watson branching process. An application is then made to the counterexample of the author's earlier paper.

Suppose that we are given a set of n elements which are to be arranged in some order. At each unit of time a request is made to retrieve one of these elements — the ith being requested with probability Pi. We show that the rule which always moves the requested element one closer to the front of the line minimizes the average position of the element requested among a wide class of rules for all probability vectors of the form P1 = p, P2= · ·· = Pn = (1 – p)/(n − 1). We also consider the above problem when the decision-maker is allowed to utilize such rules as ‘only make a change if the same element has been requested k times in a row', and show that as k approaches infinity we can do as well as if we knew the values of the Pi.

We generalize Jackson's theorem to the non-ergodic case. Here, despite the fact that the entire Jackson network will not achieve steady state, it is still possible to determine the maximal subnetwork that does. We do so by formulating and algorithmically solving a new non-linear throughput equation. These results, together with the ergodic results and the ones for closed networks, completely characterize the large-time behavior of any Jackson network.

By considering k-out-of-n systems with independent and nonidentically distributed components, we discuss stochastic monotone properties of the residual life and the inactivity time. We then present some stochastic comparisons of two systems based on the residual life and inactivity time.

Let p denote the class of all probability measures defined on the Borel subsets of the unit interval I = [0, 1]. For each positive integer n, take Mn is convex, closed, bounded, and n-dimensional; the convex hull of the space curve {(t,t2, …, tn): 0 ≦ t ≦ 1}; e.g., see Theorems 7.2, 7.3 of [1]. At each point (c1, C2, …, cn) of Mn, define Note that v−, v+ depend only on C1, C2, …, Cn− 1; Vm only on cn; We shall as notational convenience dictates and as will be apparent from the context regard v±n as functions on Mn− 1 or on higher order moment spaces and also regard Vn as a function on moment spaces of order higher than n.

In this article, we assume that the state of a system forms a continuous-time Markov chain or a higher-dimensional Markov process after introducing some supplementary variables. A formula for evaluating the rate of occurrence of failures for the system is derived. As an application of the theory, a maintenance model for a two-component system is also studied.

A system is subject to a sequence of shocks occurring randomly at times n = 1, 2, ···; each shock causes a random amount of damage. The system might fail at any point in time n, and the probability of a failure depends on the history of the system. Upon failure the system is replaced by a new and identical system and a cost is incurred. If the system is replaced before failure a smaller cost is incurred. We study the problem of specifying a replacement rule which minimizes the long-run (expected) average cost per unit time. A special case, in which the system fails when the total damage first exceeds a fixed threshold, is analysed in detail.