This paper considers the average optimality for a continuous-time Markov decision process in Borel state and action spaces, and with an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is proved under the conditions that allow the following; the controlled process can be explosive, the transition rates are weakly continuous, and the multifunction defining the admissible action spaces can be neither compact-valued nor upper semicontinuous.

For many practical situations in reliability engineering, components in the system are usually dependent since they generally work in a collaborative environment. In this paper we build sufficient conditions for comparing two coherent systems under different random environments in the sense of the usual stochastic, hazard rate, reversed hazard rate, and likelihood ratio orders. Applications and numerical examples are provided to illustrate all the theoretical results established here.

The spectral factorization problem, i.e. the problem of obtaining all possible MA representations of a process with given autocovariance function, is considered for univariate, d-periodic MA(1) (equivalently, 1-dependent in the second-order sense) processes. The solutions are provided explicitly, and their invertibility properties are investigated. A characterization, in terms of their autocovariance functions, of non-invertible d-periodic 1-dependent processes, extending to the periodic case the traditional unit root condition, is provided.

The Samaniego signature is a relevant tool for studying the performance of a system whose component lifetimes are exchangeable. It is well known that the stochastic ordering of the signatures of two systems implies the same for the respective system lifetimes. We prove that the reverse claim is not true when the component lifetimes are independent and identically distributed. There exist small proportions of systems with stochastically ordered lifetimes whose signatures are not ordered. We present a simple procedure in order to check whether the system lifetimes are stochastically ordered even if their signatures are not comparable.

We consider long-run averages of additive functionals on infinite discrete-state Markov chains, either continuous or discrete in time. Special cases include long-run average costs or rewards, stationary moments of the components of ergodic multi-dimensional Markov chains, queueing network performance measures, and many others. By exploiting Foster-Lyapunov-type criteria involving drift conditions for the finiteness of long-run averages we determine suitable finite subsets of the state space such that the truncation error is bounded. Illustrative examples demonstrate the application of this method.

We consider a random walks system on ℤ in which each active particle performs a nearest-neighbor random walk and activates all inactive particles it encounters. The movement of an active particle stops when it reaches a certain number of jumps without activating any particle. We prove that if the process relies on efficient particles (i.e. those particles with a small probability of jumping to the left) being placed strategically on ℤ, then it might survive, having active particles at any time with positive probability. On the other hand, we may construct a process that dies out eventually almost surely, even if it relies on efficient particles. That is, we discuss what happens if particles are initially placed very far away from each other or if their probability of jumping to the right tends to 1 but not fast enough.

Many prominent continuous-time stochastic volatility models exhibit certain functional relationships between price jumps and volatility jumps. We show that stochastic volatility models like the Ornstein–Uhlenbeck and other continuous-time CARMA models as well as continuous-time GARCH and EGARCH models all exhibit such functional relations. We investigate the asymptotic behaviour of certain functionals of price and volatility processes for discrete observations of the price process on a grid, which are relevant for estimation and testing problems.

It is well known that the central limit theorem holds for partial sums of a stationary sequence (Xi) of m-dependent random variables with finite variance; however, the limit may be degenerate with variance 0 even if var(Xi) ≠ 0. We show that this happens only in the case when Xi – EXi = Yi – Yi–1 for an (m − 1)-dependent stationary sequence (Yi) with finite variance (a result implicit in earlier results), and give a version for block factors. This yields a simple criterion that is a sufficient condition for the limit not to be degenerate. Two applications to subtree counts in random trees are given.

Reaction networks are commonly used within the mathematical biology and mathematical chemistry communities to model the dynamics of interacting species. These models differ from the typical graphs found in random graph theory since their vertices are constructed from elementary building blocks, i.e. the species. We consider these networks in an Erdös–Rényi framework and, under suitable assumptions, derive a threshold function for the network to have a deficiency of zero, which is a property of great interest in the reaction network community. Specifically, if the number of species is denoted by n and the edge probability by $p_n$ , then we prove that the probability of a random binary network being deficiency zero converges to 1 if $p_n\ll r(n)$ as $n \to \infty$ , and converges to 0 if $p_n \gg r(n)$ as $n \to \infty$ , where $r(n)=\frac{1}{n^3}$ .

A general method is developed to compute easy bounds of the weighted stationary probabilities for networks of queues which do not satisfy the standard product form. The bounds are obtained by constructing approximating reversible Markov chains. Thus, the bounds are insensitive with respect to service-time distributions. A special representation, called the tree-form solution, of the stationary distribution is used to derive the bounds. The results are applied to an overflow model.

A technique is developed for the analysis of the asymptotic behaviour in the long run of queueing systems with two waiting lines. The generating function of the time-dependent joint queue-length distribution is obtained with the aid of the theory of boundary value problems of the Riemann–Hilbert type and by introducing a conformal mapping of the unit disk onto a given domain. In the asymptotic analysis an extensive use is made of theorems on the boundary behaviour of such conformal mappings.

We consider a system where jobs are processed by parallel machines. The processing times are exponentially distributed. An essential feature is that the assignment of the jobs to the machines is decided before the system starts to work. We consider both the flow time and the makespan. In the case of the flow time we allow both the machines and the jobs to be non-homogeneous. The optimization is by minimizing the flow time in the sense of stochastic order and the optimal assignment is obtained for this case. The case of the makespan is harder. We consider the expected makespan and as a partial solution we prove an optimality result for the case where there are two non-homogeneous machines and the jobs are homogeneous. It turns out that the optimal assignment can be expressed by using a quantile of a binomial distribution.

Considering a random binary tree with n labelled leaves, we use a pruning procedure on this tree in order to construct a β(3/2,1/2)-coalescent process. We also use the continuous analogue of this construction, i.e. a pruning procedure on Aldous's continuum random tree, to construct a continuous state space process that has the same structure as the β-coalescent process up to some time change. These two constructions enable us to obtain results on the coalescent process, such as the asymptotics on the number of coalescent events or the law of the blocks involved in the last coalescent event.

The multitype Galton-Watson process is considered both with and without immigration. Proofs are given for the existence of invariant measures and their uniqueness is examined by functional equation methods. Theorem 2.1 proves the uniqueness, under certain conditions, of solutions of a multidimensional Schröder equation. Regular variation is shown to play a role in the multitype theory.

In the theory of autoregressive model fitting it is of interest to know the asymptotic behaviour, for large sample size, of the coefficients fitted. A significant role is played in this connection by the moments of the norms of the inverse sample covariance matrices. We establish uniform boundedness results for these, first under generally weak conditions and then for the special case of (infinite order) processes. These in turn imply corresponding ergodic theorems for the matrices in question.

A dynamic programming approach for the investigation of learning systems is taken. Making use of one-stage decision models and dynamic programs, respectively, two learning models are formulated and the existence of optimal strategies for learning in the respective models is proven.

It is generally recognized that Alfred Lotka made the first application of standard Galton-Watson branching process theory to calculate an extinction probability in a specific population (using asexual reproduction). This note applies bisexual Galton-Watson branching process theory to the calculation of an extinction probability from Lotka's data, yielding a somewhat higher value.

Let {X(t), N(t)}, – ∞< t <∞, be a stationary bivariate stochastic process where X(t) is an ordinary time series and N(t) is an orderly point process counting the number of points in (0, t]. Suppose values of {X(t), N(t)} are available for 0< t ≦T and let σ1, σ2, ···, σ N(T) denote the jump points of N(t) in (0, T]. For |v| < T, define mT12(υ)=∑ X (σj+υ)/T and μT12(υ)=∑ X (σj+υ)/∑1 where all summations are over indices j such that 0<σj, σj+υ≦T for some σj. The functions MT12(υ) and μT12(υ) are often useful in analyzing the covariation of the time series and point process. In this paper, we shall develop some statistical properties of the functions MT12(υ) and μT12(υ) and discuss some specific situations where it is useful to consider these functions.

Two people independently and with the same distribution guess the location of an unseen object in n-dimensional space, and the one whose guess is closer to the unseen object is declared the winner. The first person announces his guess, but the second modifies his unspoken idea by moving his guess in the direction of the first guess and as close to it as possible. It is shown that if the distribution of guesses is rotationally symmetric about the true location of the unseen object, ¾ is the sharp lower bound for the success probability of the second guesser. If the distribution is fixed and the dimension increases, then for a certain class of distributions, the success probability approaches 1.

In a recent paper (Downton (1972)) an attempt was made to use the combinatorial approach of Downton (1967) to obtain an expression for the Laplace transform (and hence the moments) of the area under the infectives trajectory of the general stochastic epidemic. The basic property of that area, which makes this possible, is that it is composed of a random number of rectangles, each of whose area has a distribution depending on the infection rate and the number of susceptibles present at the appropriate stage in the development of the epidemic, but which is independent of the number of infectives. However, Mr A. Abakuks has pointed out that in one respect the argument used was faulty. It was argued that the contribution to the area under the infectives curve of an interval ending in a reduction in the number of infectives from i to i – 1 had an exponential distribution with parameter p (the infection rate); and if the interval ended in a reduction in the number of susceptibles from s to s – 1 the distribution was exponential with parameter s. While this is true unconditionally, the combinatorial situation described was essentially a conditional one, in which each of these contributions to the total area had an exponential distribution, independent of i, but with parameter (ρ + s) regardless of whether the interval ended in a reduction in the number of infectives or of susceptibles.