Parrondo’s coin-tossing games were introduced as a toy model of the flashing Brownian ratchet in statistical physics but have emerged as a paradigm for a much broader phenomenon that occurs if there is a reversal in direction in some system parameter when two similar dynamics are combined. Our focus here, however, is on the original Parrondo games, usually labeled A and B. We show that if the parameters of the games are allowed to be arbitrary, subject to a fairness constraint, and if the two (fair) games A and B are played in an arbitrary periodic sequence, then the rate of profit can not only be positive (the so-called Parrondo effect), but can also be arbitrarily close to 1 (i.e. 100%).

We examine necessary and sufficient conditions for recurrence and positive recurrence of a class of irreducible, level-dependent quasi-birth-and-death (LDQBD) processes with a block tridiagonal structure that exhibits asymptotic convergence in the rows as the level tends to infinity. These conditions are obtained by exploiting a multi-dimensional Lyapunov drift approach, along with the theory of generalized Markov group inverses. Additionally, we highlight analogies to well-known average drift results for level-independent quasi-birth-and-death (QBD) processes.

The aim of this paper is to prove an inequality between relative entropy and the sum of average conditional relative entropies of the following form: for a fixed probability measure q on , ( is a finite set), and any probability measure on , (*)

$$D(p||q){\rm{\le}}C \cdot \sum\limits_{i = 1}^n {{\rm{\mathbb{E}}}_p D(p_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,...,{\rm{ }}Y_n )||q_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,{\rm{ }}...,{\rm{ }}Y_n )),} $$

The inequality (*) ismeaningful in product spaces, in both the discrete and the continuous case, and can be used to prove a logarithmic Sobolev inequality for q, provided uniform logarithmic Sobolev inequalities are available for qi(· |x1, …, xi−1, xi+1, …, xn), for all fixed i and fixed (x1, …, xi−1, xi+1, …, xn). Inequality (*) directly implies that the Gibbs sampler associated with q is a contraction for relative entropy.

In this paper we derive inequality (*), and thereby a logarithmic Sobolev inequality, in discrete product spaces, by proving inequalities for an appropriate Wasserstein-like distance.

We show that a coupling of non-colliding simple random walkers on the complete graph on n vertices can include at most n - log n walkers. This improves the only previously known upper bound of n - 2 due to Angel, Holroyd, Martin, Wilson and Winkler (Electron. Commun. Probab.18 (2013)). The proof considers couplings of i.i.d. sequences of Bernoulli random variables satisfying a similar avoidance property, for which there is separate interest.

In their 1960 book on finite Markov chains, Kemeny and Snell established that a certain sum is invariant. The value of this sum has become known as Kemeny’s constant. Various proofs have been given over time, some more technical than others. We give here a very simple physical justification, which extends without a hitch to continuous-time Markov chains on a finite state space. For Markov chains with denumerably infinite state space, the constant may be infinite and even if it is finite, there is no guarantee that the physical argument will hold. We show that the physical interpretation does go through for the special case of a birth-and-death process with a finite value of Kemeny’s constant.

Let (Mn,Sn)n≥0 be a Markov random walk with positive recurrent driving chain (Mn)n≥0 on the countable state space 𝒮 with stationary distribution π. Suppose also that lim supn→∞Sn=∞ almost surely, so that the walk has almost-sure finite strictly ascending ladder epochs σn>. Recurrence properties of the ladder chain (Mσn>)n≥0 and a closely related excursion chain are studied. We give a necessary and sufficient condition for the recurrence of (Mσn>)n≥0 and further show that this chain is positive recurrent with stationary distribution π> and 𝔼π>σ1><∞ if and only if an associated Markov random walk (𝑀̂n,𝑆̂n)n≥0, obtained by time reversal and called the dual of (Mn,Sn)n≥0, is positive divergent, i.e. 𝑆̂n→∞ almost surely. Simple expressions for π> are also provided. Our arguments make use of coupling, Palm duality theory, and Wiener‒Hopf factorization for Markov random walks with discrete driving chain.

We give the first polynomial upper bound on the mixing time of the edge-flip Markov chain for unbiased dyadic tilings, resolving an open problem originally posed by Janson, Randall and Spencer in 2002 [14]. A dyadic tiling of size n is a tiling of the unit square by n non-overlapping dyadic rectangles, each of area 1/n, where a dyadic rectangle is any rectangle that can be written in the form [a2−s, (a + 1)2−s] × [b2−t, (b + 1)2−t] for a, b, s, t ∈ ℤ⩾ 0. The edge-flip Markov chain selects a random edge of the tiling and replaces it with its perpendicular bisector if doing so yields a valid dyadic tiling. Specifically, we show that the relaxation time of the edge-flip Markov chain for dyadic tilings is at most O(n4.09), which implies that the mixing time is at most O(n5.09). We complement this by showing that the relaxation time is at least Ω(n1.38), improving upon the previously best lower bound of Ω(n log n) coming from the diameter of the chain.

We consider upper‒lower (UL) (and lower‒upper (LU)) factorizations of the one-step transition probability matrix of a random walk with the state space of nonnegative integers, with the condition that both upper and lower triangular matrices in the factorization are also stochastic matrices. We provide conditions on the free parameter of the UL factorization in terms of certain continued fractions such that this stochastic factorization is possible. By inverting the order of the factors (also known as a Darboux transformation) we obtain a new family of random walks where it is possible to state the spectral measures in terms of a Geronimus transformation. We repeat this for the LU factorization but without a free parameter. Finally, we apply our results in two examples; the random walk with constant transition probabilities, and the random walk generated by the Jacobi orthogonal polynomials. In both situations we obtain urn models associated with all the random walks in question.

Let P be the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vector πT, and let (n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n + 1) x (n + 1) northwest corner truncation of P arbitrarily, with invariant probability vector (n)πT. We derive computable V-norm bounds on the error between πT and (n)πT in terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 as n tends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

In this paper we discuss the Mabinogion urn model introduced by Williams (1991). Therein he describes an optimal control problem where the objective is to maximize the expected final number of objects of one kind in the Mabinogion urn model. Our main contribution is formulae for the expected time to absorption and its asymptotic behaviour in the optimally controlled process. We also present results for the noncontrolled Mabinogion urn process and briefly analyze other strategies that become superior if a certain discount factor is included.

We construct a simple example, surely known to Harry Kesten, of an R-transient Markov chain on a countable state space S ∪ {δ}, where δ is absorbing. The transition matrix K on S is irreducible and strictly substochastic. We determine the Yaglom limit, that is, the limiting conditional behavior given nonabsorption. Each starting state x ∈ S results in a different Yaglom limit. Each Yaglom limit is an R-1-invariant quasi-stationary distribution, where R is the convergence parameter of K. Yaglom limits that depend on the starting state are related to a nontrivial R-1-Martin boundary.

Given a sequence (Mk, Qk)k ≥ 1 of independent and identically distributed random vectors with nonnegative components, we consider the recursive Markov chain (Xn)n ≥ 0, defined by the random difference equation Xn = MnXn - 1 + Qn for n ≥ 1, where X0 is independent of (Mk, Qk)k ≥ 1. Criteria for the null recurrence/transience are provided in the situation where (Xn)n ≥ 0 is contractive in the sense that M1 ⋯ Mn → 0 almost surely, yet occasional large values of the Qn overcompensate the contractive behavior so that positive recurrence fails to hold. We also investigate the attractor set of (Xn)n ≥ 0 under the sole assumption that this chain is locally contractive and recurrent.

In this paper we study the random walk on the hypercube (ℤ / 2ℤ)n which at each step flips k randomly chosen coordinates. We prove that the mixing time for this walk is of the order (n / k)logn. We also prove that if k = o(n) then the walk exhibits cutoff at (n / 2k)logn with window n / 2k.

We observe the frog model, an infinite system of interacting random walks, on ℤ with an asymmetric underlying random walk. For certain initial frog distributions we construct an explicit formula for the moments of the leftmost visited site, as well as their asymptotic scaling limits as the drift of the underlying random walk vanishes. We also provide conditions in which the lower bound can be scaled to converge in probability to the degenerate distribution at 1 as the drift vanishes.

The infinite-bin model, introduced by Foss and Konstantopoulos (2003), describes the Markovian evolution of configurations of balls placed inside bins, obeying certain transition rules. We prove that we can couple the behaviour of any finite number of balls, provided at least two different transition rules are allowed. This coupling makes it possible to define the regeneration events needed by Foss and Zachary (2013) to prove convergence results for the distribution of the balls.

We introduce and study a new model that we call the matching model. Items arrive one by one in a buffer and depart from it as soon as possible but by pairs. The items of a departing pair are said to be matched. There is a finite set of classes 𝒱 for the items, and the allowed matchings depend on the classes, according to a matching graph on 𝒱. Upon arrival, an item may find several possible matches in the buffer. This indeterminacy is resolved by a matching policy. When the sequence of classes of the arriving items is independent and identically distributed, the sequence of buffer-content is a Markov chain, whose stability is investigated. In particular, we prove that the model may be stable if and only if the matching graph is nonbipartite.

We analyse the 𝓁²(𝜋)-convergence rate of irreducible and aperiodic Markov chains with N-band transition probability matrix P and with invariant distribution 𝜋. This analysis is heavily based on two steps. First, the study of the essential spectral radius r ess(P |𝓁²(𝜋)) of P |𝓁²(𝜋) derived from Hennion’s quasi-compactness criteria. Second, the connection between the spectral gap property (SG2) of P on 𝓁²(𝜋) and the V-geometric ergodicity of P. Specifically, the (SG2) is shown to hold under the condition α0≔∑m=−N N lim supi→+∞(P(i,i+m)P *(i+m,i)1∕2<1. Moreover, r ess(P |𝓁²(𝜋)≤α0. Effective bounds on the convergence rate can be provided from a truncation procedure.

We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

Let I be a finite set and S be a nonempty strict subset of I which is partitioned into classes, and let C(s) be the class containing s ∈ S. Let (Ps: s ∈ S) be a family of distributions on IN, where each Ps applies to sequences starting with the symbol s. To this family, we associate a class of distributions P(π) on IN which depends on a probability vector π. Our main results assume that, for each s ∈ S, Ps regenerates with distribution Ps' when it encounters s' ∈ S ∖ C(s). From semiregenerative theory, we determine a simple condition on π for P(π) to be time stationary. We give a similar result for the following more complex model. Once a symbol s' ∈ S ∖ C(s) has been encountered, there is a decision to be made: either a new region of type C(s') governed by Ps' starts or the region continues to be a C(s) region. This decision is modeled as a random event and its probability depends on s and s'. The aim in studying these kinds of models is to attain a deeper statistical understanding of bacterial DNA sequences. Here I is the set of codons and the classes (C(s): s ∈ S) identify codons that initiate similar genomic regions. In particular, there are two classes corresponding to the start and stop codons which delimit coding and noncoding regions in bacterial DNA sequences. In addition, the random decision to continue the current region or begin a new region of a different class reflects the well-known fact that not every appearance of a start codon marks the beginning of a new coding region.

Let N and M be positive integers satisfying 1≤ M≤ N, and let 0< p0 < p1 < 1. Define a process {Xn}n=0∞ on ℤ as follows. At each step, the process jumps either one step to the right or one step to the left, according to the following mechanism. For the first N steps, the process behaves like a random walk that jumps to the right with probability p0 and to the left with probability 1-p0. At subsequent steps the jump mechanism is defined as follows: if at least M out of the N most recent jumps were to the right, then the probability of jumping to the right is p1; however, if fewer than M out of the N most recent jumps were to the right then the probability of jumping to the right is p0. We calculate the speed of the process. Then we let N→ ∞ and M/N→ r∈[0,1], and calculate the limiting speed. More generally, we consider the above questions for a random walk with a finite number l of threshold levels, (Mi,pi) i=1l, above the pre-threshold level p0, as well as for one model with l=N such thresholds.