In this paper, a fourth moment bound for partial sums of functionals of strongly ergodic Markov chains is established. This type of inequality plays an important role in the study of the empirical process invariance principle. This inequality is specially adapted to the technique of Dehling, Durieu, and Volný (2008). The same moment bound can be proved for dynamical systems whose transfer operator has some spectral properties. Examples of applications are given.

Gene conversion is a genetic mechanism by which one gene is ‘copied and pasted’ onto another gene, where the direction can be biased between the different types. In this paper, a stochastic model for biased gene conversion within a d-unlinked multigene family and its diffusion approximation are developed for a finite Moran population. A connection with a d-island model is made. A formula for the fixation probability in the absence of mutation is given. A two-timescale argument is applied in the case of the strong conversion limit. The dual process is generally shown to be a biased voter model, which generates an ancestral bias graph for a given sample. An importance sampling algorithm for computing the likelihood of the sample is deduced.

We consider backward stochastic differential equations in a setting where noise is generated by a countable state, continuous time Markov chain, and the terminal value is prescribed at a stopping time. We show that, given sufficient integrability of the stopping time and a growth bound on the terminal value and BSDE driver, these equations admit unique solutions satisfying the same growth bound (up to multiplication by a constant). This holds without assuming that the driver is monotone in y, that is, our results do not require that the terminal value be discounted at some uniform rate. We show that the conditions are satisfied for hitting times of states of the chain, and hence present some novel applications of the theory of these BSDEs.

A lumping of a Markov chain is a coordinatewise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, if and only if the lumped process is a kth-order Markov chain for each starting distribution of the original Markov chain. We characterise strong k-lumpability via tightness of stationary entropic bounds. In the sparse setting, we give sufficient conditions on the lumping to both preserve the entropy rate and be strongly k-lumpable.

We obtain a representation of an arbitrary discrete distribution with n mass points by an equiprobable mixture of r distributions, each of which has no more than a (≧2) mass points, where r is the smallest integer greater than or equal to (n – 1)/(a – 1). An application to the generation of discrete random variables on a computer is described, which has as an important special case Walker's (1977) alias method.

We consider an M/GI/1 queue with two types of customers, positive and negative, which cancel each other out. The server provides service to either a positive customer or a negative customer. In such a system, the queue length can be either positive or negative and an arrival either joins the queue, if it is of the same sign, or instantaneously removes a customer of the opposite sign at the end of the queue or in service. This study is a generalization of Gelenbe's original concept of a queue with negative customers, where only positive customers need services and negative customers arriving at an empty system are lost or need no service. In this paper, we derive the transient and the stationary probability distributions for the major performance measures in terms of generating functions and Laplace transforms. It has been shown that the previous results for the system with negative arrivals of zero service time are special cases of our model. In addition, we obtain the stationary waiting time distribution of this system in terms of a Laplace transform.

The paper characterizes matrices which have a given system of vectors orthogonal with respect to a given probability distribution as its right eigenvectors. Results of Hoare and Rahman are unified in this context, then all matrices with a given orthogonal polynomial system as right eigenvectors under the constraint a0j = 0 for j ≥ 2 are specified. The only stochastic matrices P = {pij} satisfying p00 + p01 = 1 with the Hahn polynomials as right eigenvectors have the form of the Moran mutation model.

Stochastic dynamical system models for non-linear random vibrations are considered and their discrete-time version, non-linear time series models are introduced using the local Gaussian modelling method. Some computational problems and implications of the present method in non-linear time series analysis are discussed.

In this paper, we introduce a compound random mapping model which can be viewed as a generalization of the basic random mapping model considered by Ross and by Jaworski. We investigate a particular example, the Poisson compound random mapping, and compare results for this model with results known for the well-studied uniform random mapping model. We show that, although the structure of the components of the random digraph associated with a Poisson compound mapping differs from the structure of the components of the random digraph associated with the uniform model, the limiting distribution of the normalized order statistics for the sizes of the components is the same as in the uniform case, i.e. the limiting distribution is the Poisson-Dirichlet (½) distribution on the simplex {{xi} : ∑ xi ≤ 1, xi ≥ xi+1 ≥ 0 for every i ≥ 1}.

We consider the Klimov model for an open network of two types of jobs. Jobs of type i arrive at station i, have processing times that are exponentially distributed with parameter µi, and when processed either go on to station j with probability pij, or depart the network with probability pi0. Costs are charged at a rate that depends on the number of jobs of the two types in the system. It is shown that for arbitrary arrival processes the policy that gives priority to those jobs for whom the rate of change of the cost function is greatest minimizes the expected cost rate at every time t. This result is stronger than the Klimov result in two ways: arrival processes are arbitrary, and the minimization is at each time t. But the result holds for only two types.

Let n points be distributed independently within a k-dimensional unit cube according to density f. At each point, construct a k-dimensional sphere of content an. Let V denote the vacancy, or ‘volume' not covered by the spheres. We derive asymptotic formulae for the mean and variance of V, as n → ∞and an → 0. The formulae separate naturally into three cases, corresponding to nan → 0, nan → a (0 < a <∞) and nan →∞, respectively. We apply the formulae to derive necessary and sufficient conditions for V/E(V) → 1 in L2.

In the multi-armed bandit problem, the decision-maker must choose each period a single project to work on. From the chosen project she receives an immediate reward that depends on the current state of the project. Next period the chosen project makes a stochastic transition to a new state, but projects that are not chosen remain in the same state. What happens in a two-armed bandit context if projects not chosen do not remain in the same state? We derive two sufficient conditions for the optimal policy to be myopic: either the transition function for chosen projects has in a certain sense uniformly stronger stochastic dominance than the transition function for unchosen projects, or both transition processes are normal martingales, the variance of which is independent of the history of process choices.

Here we consider the Kohonen algorithm with a constant learning rate as a Markov process evolving in a topological space. Despite the fact that the algorithm is not weak Feller, we show that it is a T-chain, regardless of the dimensionalities of both data space and network and the special shape of the neighborhood function. In addition for the practically important case of the multi-dimensional setting, it is shown that the chain is irreducible and aperiodic. We show that these imply the validity of Doeblin's condition, which in turn ensures the convergence in distribution of the process to an invariant probability measure with a geometric rate. Furthermore, it is shown that the process is positive Harris recurrent, which enables us to use statistical devices to measure the centrality and variability of the invariant probability measure. Our results cover a wide class of neighborhood functions.

The response of the photographic grain to light is a non-deterministic process which is as yet not completely understood. This response, as measured by the photographic density, is usually taken to be a function of the product of incident light intensity and exposure time interval duration, but at extreme values of either of these two quantities, this is no longer true. This latter effect is called reciprocity-law failure.

This paper discusses a probabilistic model, similar to a multiserver queue, for high-intensity reciprocity failure.

Consider a sequence of conditionally independent Bernoulli random variables taking on the values 1 and − 1. The objective is to stop the sequence in order to maximize the discounted sum. Suppose the Bernoulli parameter has a beta distribution with integral parameters. It is optimal to stop when the conditional expectation of the next random variable is negative provided the discount factor is less than or equal to . Moreover, is best possible. The case where the parameters of the beta distribution are arbitrary positive numbers is also treated.

We prove a ratio identity giving a transform of the distribution of the position of a two barrier random walk as a ratio of the equivalent transforms for the one barrier random walk. This ratio identity holds for skip free random walks where the increments are dependent on both the previous increment and also the position of the random walk.

In this paper the integrated three-valued telegraph process is examined. In particular, the third-order equations governing the distributions , (where N(t) denotes the number of changes of the telegraph process up to time t) are derived and recurrence relationships for them are obtained by solving suitable initial-value problems. These recurrence formulas are related to the Fourier transform of the conditional distributions and are used to obtain explicit results for small values of k. The conditional mean values (where V(0) denotes the initial velocity of motions) are obtained and discussed.

Let a+ = a if a > 0, a+ = 0 otherwise, and a– = (– a)+.

We consider two random sequential packing processes in which spheres of unit radius are randomly attached to the surface of a fixed unit sphere. Independent random spheres are generated and added successively, provided there is no overlap with previous spheres. In model 1, the process stops when a trial sphere intersects one of the previously-accepted spheres. In model 2, random sequential packing, any such overlapping trial sphere is discarded and the next random sphere is tried, until it is impossible to add any further spheres.

Previous workers have conjectured convincingly that no exact analytical solution is possible for this type of problem. We use Monte Carlo simulation methods to estimate transition probabilities for the two models. Because some probabilities are extremely small, a simulation using independent repetitions of the model would be inefficient. We designed a branching process of conditionally binomial trials, and performed over 108 trials on a supercomputer.

In this paper we are interested in a nonlinear parabolic evolution equation occurring in rheology. We give a probabilistic interpretation to this equation by associating a nonlinear martingale problem with it. We prove the existence of a unique solution, P, to this martingale problem. For any t, the time marginal of P at time t admits a density ρ(t,x) with respect to the Lebesgue measure, where the function ρ is the unique weak solution to the evolution equation in a well-chosen energy space. Next we introduce a simulable system of n interacting particles and prove that the empirical measure of this system converges to P as n tends to ∞. This propagation-of-chaos result ensures that the solution to the equation of interest can be approximated using a Monte Carlo method. Finally, we illustrate the convergence in some numerical experiments.