This work concerns controlled Markov chains with denumerable state space, (possibly) unbounded cost function, and an expected average cost criterion. Under a Lyapunov function condition, together with mild continuity-compactness assumptions, a simple necessary and sufficient criterion is given so that the relative value functions and differential costs produced by the value iteration scheme converge pointwise to the solution of the optimality equation; this criterion is applied to obtain convergence results when the cost function is bounded below or bounded above.

We construct a model of mutually non-intersecting balls using a generalized notion of Gibbsian processes and characterize its size, i.e. radius, distribution. The explicit form of this distribution is given in the one-dimensional case, that is, mutually non-intersecting segment processes on the line.

In this paper, we give necessary and sufficient conditions to ensure the validity of confidence intervals, based on the central limit theorem, in simulations of highly reliable Markovian systems. We resort to simulations because of the frequently huge state space in practical systems. So far the literature has focused on the property of bounded relative error. In this paper we focus on ‘bounded normal approximation’ which asserts that the approximation of the normal law, suggested by the central limit theorem, does not deteriorate as the reliability of the system increases. Here we see that the set of systems with bounded normal approximation is (strictly) included in the set of systems with bounded relative error.

We define a leavable stochastic game which is a possible two-person generalization of the classical red-and-black gambling problem. We show that there are three basic possibilities for a two-person red-and-black game which, by analogy with gambling theory, we call the subfair, the fair and the superfair cases. A suitable generalization of what in gambling theory is called bold play is proved to be a uniformly ε-optimal stationary strategy for player I in the fair and the subfair cases whereas a generalization of timid play is shown to be ε-optimal for player I in the superfair possibility.

In this paper we describe a model for survival functions. Under this model a system is subject to shocks governed by a Poisson process. Each shock to the system causes a random damage that grows in time. Damages accumulate additively and the system fails if the total damage exceeds a certain capacity or threshold. Various properties of this model are obtained. Sufficient conditions are derived for the failure rate (FR) order and the stochastic order to hold between the random lifetimes of two systems whose failures can be described by our proposed model.

The conjecture pronounced at the end of the paper of Srivastava and Srivastava (1970) is proved in this paper. It gives the following characterization of (bivariate) Poisson distributions. Suppose that items of two types have been observed certain numbers of times, but these original observations have been reduced due to a destructive process which is the product of two binomial distributions and that the probabilities of these reduced numbers are the same whether damaged or undamaged. Then the original random variables had a bivariate Poisson distribution with zero mutual dependence coefficient.

Sankoff and Ferretti (1996) introduced several models of the evolution of chromosome size by reciprocal translocations, where for simplicity they ignored the existence of centromeres. However, when they compared the models to data on six organisms they found that their short chromosomes were too short, and their long chromosomes were too long. Here, we consider a generalization of their proportional model with explicit chromosome centromeres and introduce fitness functions based on recombination probabilities and on the length of the longest chromosome arm. We find a simple formula for the stationary distribution for our model which fits the data on chromosome lengths in many, but not all, species.

Keilson [7] showed that for a birth-death process defined on non-negative integers with reflecting barrier at 0 the first-passage-time density from 0 to N (N to N + 1) has Pólya frequency of order infinity (is completely monotone). Brown and Chaganty [3] and Assaf et al. [1] studied the first-passage-time distribution for classes of discrete-time Markov chains and then produced the essentially same results as these through a uniformization. This paper addresses itself to an extension of Keilson's results to classes of Markov chains such as time-reversible Markov chains, skip-free Markov chains and birth-death processes with absorbing states. The extensions are due to the spectral representations of the infinitesimal generators governing these Markov chains. Explicit densities for those first-passage times are also given.

Our main concern in this paper is the convergence, as t → ∞, of the quantities i, j ∈ E; where Pij(t) is the transition probability of a semi-Markov process whose state space E is irreducible but not closed (i.e., escape from E is possible), and rj is the probability of eventual escape from E conditional on the initial state being i. The theorems proved here generalize some results of Seneta and Vere-Jones ([8] and [11]) for Markov processes.

Using the characterization of point processes having the order statistic property we prove that the only renewal process that has the order statistic property is the Poisson process.

Through the study of a simple embedded martingale we obtain an extension of the Kesten–Stigum theorem and prove a central limit theorem for controlled Galton-Watson processes.

By lower estimates of the functionals 𝔼[eStKtNt], where St and Nt denote the total length up to time t and the number of individuals at time t in a Galton-Watson tree, we obtain sufficient criteria for the blow-up of semilinear equations and systems of the type ∂wt/∂t = Awt + Vwtβ. Roughly speaking, the growth of the tree length has to win against the ‘mobility’ of the motion belonging to the generator A, since, in the probabilistic representation of the equations, the latter results in small K(t) as t → ∞. In the single-type situation, this gives a re-interpretation of classical results of Nagasawa and Sirao(1969); in the multitype scenario, part of the results obtained through analytic methods in Escobedo and Herrero (1991) and (1995) are re-proved and extended from the case A = Δ to the case of α-Laplacians.

We study the optimal portfolio problem for an insider, in the case where the performance is measured in terms of the logarithm of the terminal wealth minus a term measuring the roughness and the growth of the portfolio. We give explicit solutions in some cases. Our method uses stochastic calculus of forward integrals.

Let X be a pure-jump subordinator (i.e. nondecreasing Lévy process with no drift) with infinite Lévy measure, let Xε be the sum of jumps not exceeding ε, and let µ(ε)=E[Xε(1)]. We study the question of weak convergence of Xε/µ(ε) as ε ↓0, in terms of the limit behavior of µ(ε)/ε. The most interesting case reduces to the weak convergence of Xε/ε to a subordinator whose marginals are generalized Dickman distributions; we give some necessary and sufficient conditions for this to hold. For a certain significant class of subordinators for which the latter convergence holds, and whose most prominent representative is the gamma process, we give some detailed analysis regarding the convergence quality (in particular, in the context of approximating X itself). This paper completes, in some respects, the study made by Asmussen and Rosiński (2001).

We study a Markovian agent-based model (MABM) in this paper. Each agent is endowed with a local state that changes over time as the agent interacts with its neighbours. The neighbourhood structure is given by a graph. Recently, Simon, Taylor, and Kiss [40] used the automorphisms of the underlying graph to generate a lumpable partition of the joint state space, ensuring Markovianness of the lumped process for binary dynamics. However, many large random graphs tend to become asymmetric, rendering the automorphism-based lumping approach ineffective as a tool of model reduction. In order to mitigate this problem, we propose a lumping method based on a notion of local symmetry, which compares only local neighbourhoods of vertices. Since local symmetry only ensures approximate lumpability, we quantify the approximation error by means of the Kullback–Leibler divergence rate between the original Markov chain and a lifted Markov chain. We prove the approximation error decreases monotonically. The connections to fibrations of graphs are also discussed.

Suppose that a system consists of n independent and identically distributed components and that the life lengths of the n components are Xi, i = 1, …, n. For k ∈ {1, …, n - 1}, let X(k)1, …, X(k)n-k be the residual life lengths of the live components following the kth failure in the system. In this paper we extend various stochastic ordering results presented in Bairamov and Arnold (2008) on the residual life lengths of the live components in an (n - k + 1)-out-of-n system, and also present a new result concerning the multivariate stochastic ordering of live components in the two-sample situation. Finally, we also characterize exponential distributions under a weaker condition than those introduced in Bairamov and Arnold (2008) and show that some special ageing properties of the original residual life lengths get preserved by residual life lengths.

We consider a Poisson cluster model, motivated by insurance applications. At each claim arrival time, modeled by the point of a homogeneous Poisson process, we start a cluster process which represents the number or amount of payments triggered by the arrival of a claim in a portfolio. The cluster process is a Lévy or truncated compound Poisson process. Given the observations of the process over a finite interval, we consider the expected value of the number and amount of payments in a future time interval. We also give bounds for the error encountered in this prediction procedure.

Gabetta and Regazzini (2006b) have shown that finiteness of the initial energy (second moment) is necessary and sufficient for the solution of the Kac's model Boltzmann equation to converge weakly (Cb-convergence) to a probability measure on R. Here, we complement this result by providing a detailed analysis of what does actually happen when the initial energy is infinite. In particular, we prove that such a solution converges vaguely (C0-convergence) to the zero measure (which is identically 0 on the Borel sets of R). More precisely, we prove that the total mass of the limiting distribution splits into two equal masses (of value ½ each), and we provide quantitative estimates on the rate at which such a phenomenon takes place. The methods employed in the proofs also apply in the context of sums of weighted independent and identically distributed random variables x̃1, x̃2, …, where these random variables have an infinite second moment and zero mean. Then, with Tn := ∑j=1ηnλj,nx̃j, with max1 ≤ j ≤ ηnλj,n → 0 (as n → +∞), and ∑j=1ηnλj,n2 = 1, n = 1, 2, …, the classical central limit theorem suggests that T should in some sense converge to a ‘normal random variable of infinite variance’. Again, in this setting we prove quantitative estimates on the rate at which the mass splits into adherent masses to -∞ and +∞, or to ∞, that are analogous to those we have obtained for the Kac equation. Although the setting in this case is quite classical, we have not uncovered any previous results of a similar type.

We consider a branching population where individuals have independent and identically distributed (i.i.d.) life lengths (not necessarily exponential) and constant birth rates. We let Nt denote the population size at time t. We further assume that all individuals, at their birth times, are equipped with independent exponential clocks with parameter δ. We are interested in the genealogical tree stopped at the first time T when one of these clocks rings. This question has applications in epidemiology, population genetics, ecology, and queueing theory. We show that, conditional on {T<∞}, the joint law of (Nt, T, X(T)), where X(T) is the jumping contour process of the tree truncated at time T, is equal to that of (M, -IM, Y′M) conditional on {M≠0}. Here M+1 is the number of visits of 0, before some single, independent exponential clock e with parameter δ rings, by some specified Lévy process Y without negative jumps reflected below its supremum; IM is the infimum of the path YM, which in turn is defined as Y killed at its last visit of 0 before e; and Y′M is the Vervaat transform of YM. This identity yields an explanation for the geometric distribution of NT (see Kitaev (1993) and Trapman and Bootsma (2009)) and has numerous other applications. In particular, conditional on {NT=n}, and also on {NT=n,T<a}, the ages and residual lifetimes of the n alive individuals at time T are i.i.d. and independent of n. We provide explicit formulae for this distribution and give a more general application to outbreaks of antibiotic-resistant bacteria in the hospital.