We study the expected time to ruin in a risk process in which dividends are paid when the surplus is above the barrier. We consider the case in which the dividend rate is smaller than the premium rate. We obtain results for the classical compound Poisson risk process with phase-type claim size. When the ruin probability is 1, we derive the expected time to ruin and the expected dividends paid. When the ruin probability is less than 1, these quantities are derived conditioning on the event that ruin occurs.

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide a recursive way to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates are uniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

This work concerns Markov decision chains with finite state spaces and compact action sets. The performance index is the long-run risk-sensitive average cost criterion, and it is assumed that, under each stationary policy, the state space is a communicating class and that the cost function and the transition law depend continuously on the action. These latter data are not directly available to the decision-maker, but convergent approximations are known or are more easily computed. In this context, the nonstationary value iteration algorithm is used to approximate the solution of the optimality equation, and to obtain a nearly optimal stationary policy.

In this paper, we study a monotone process maintenance model for a multistate system with k working states and ℓ failure states. By making different assumptions, we can apply the model to a multistate deteriorating system as well as to a multistate improving system. We show that the monotone process model for a multistate system is equivalent to a geometric process model for a two-state system. Then, for both the deteriorating and the improving system, we analytically determine an optimal replacement policy for minimizing the long-run average cost per unit time.

Suppose that a gambler starts with a fortune in (0,1) and wishes to attain a fortune of 1 by making a sequence of bets. Assume that whenever the gambler stakes an amount s, the gambler's fortune increases by s with probability w and decreases by s with probability 1 − w, where w < ½. Dubins and Savage showed that the optimal strategy, which they called ‘bold play’, is always to bet min{f, 1 − f}, where f is the gambler's current fortune. Here we consider the problem in which the gambler may stake no more than ℓ at one time. We show that the bold strategy of always betting min{ℓ, f, 1 − f} is not optimal if ℓ is irrational, extending a result of Heath, Pruitt, and Sudderth.

We give a very general reformulation of multi-actor Markov decision processes and show that there is a tendency for the actors to take the same action whenever possible. This considerably reduces the complexity of the problem, either facilitating numerical computation of the optimal policy or providing a basis for a heuristic.

In this paper, we establish a simple asymptotic formula for the finite-time ruin probability of the compound Poisson model with constant interest force and subexponential claims in the case that the initial surplus is large. The formula is consistent with known results for the ultimate ruin probability and, in particular, is uniform for all time horizons when the claim size distribution is regularly varying tailed.

Consider n cells, of which some are target cells, and suppose that each cell has a weight. The cells are killed in a sequential manner, with each currently live cell being the next one killed with a probability proportional to its weight. We study the distribution of the number of cells that are alive at the moment when all the target cells have been killed.

We provide a scaling for compound Poisson distributions that leads (under certain conditions on the Fourier transform) to a weak convergence result as the parameter of the distribution tends to infinity. We show that the limiting probability measure belongs to the class of stable Cauchy laws with Fourier transform t ↦ exp(−c|t|− iat log|t|). We apply this convergence result to the standard discrete Luria–Delbrück distribution and derive an integral representation for the corresponding limiting density, as an alternative to that found in a closely related paper of Kepler and Oprea. Moreover, we verify local convergence and we derive an integral representation for the distribution function of the limiting continuous Luria–Delbrück distribution.

We investigate the conditions on a hedger, who overestimates the (time- and level-dependent) volatility, to superreplicate a convex claim on several underlying assets. It is shown that the classic Black-Scholes model is the only model, within a large class, for which overestimation of the volatility yields the desired superreplication property. This is in contrast to the one-dimensional case, in which it is known that overestimation of the volatility with any time- and level-dependent model guarantees superreplication of convex claims.

We consider epidemics in populations that are partitioned into small groups known as households. Whilst infectious, a typical infective makes global and local contact with individuals chosen independently and uniformly from the whole population or their own household, as appropriate. Previously, the classical Poisson approximation for the number of survivors of a severe epidemic has been extended to the household model. However, in the current work we exploit a Sellke-type construction of the epidemic process, which enables the derivation of sufficient conditions for the existence of a compound Poisson limit theorem for the survivors of the epidemic. The results are specialised to the Reed-Frost and general stochastic epidemic models.

We consider a stochastic model for the spread of a susceptible–infective–removed (SIR) epidemic among a closed, finite population, in which there are two types of severity of infectious individuals, namely mild and severe. The type of severity depends on the amount of infectious exposure an individual receives, in that infectives are always initially mild but may become severe if additionally exposed. Large-population properties of the model are derived. In particular, a coupling argument is used to provide a rigorous branching process approximation to the early stages of an epidemic, and an embedding argument is used to derive a strong law and an associated central limit theorem for the final outcome of an epidemic in the event of a major outbreak. The basic reproduction number, which determines whether or not a major outbreak can occur given few initial infectives, depends only on parameters of the mild infectious state, whereas the final outcome in the event of a major outbreak depends also on parameters of the severe state. Moreover, the limiting final size proportions need not even be continuous in the model parameters.

We consider a heterogeneous population of identical particles divided into a finite number of classes according to their level of health. The partition can change over time, and a suitable exchangeability assumption is made to allow for having identical items of different types. The partition is not observed; we only observe the cardinality of a particular class. We discuss the problem of finding the conditional distribution of particle lifetimes, given such observations, using stochastic filtering techniques. In particular, a discrete-time approximation is given.

The purpose of this paper is to determine the exact distribution of the final size of an epidemic for a wide class of models of susceptible–infective–removed type. First, a nonstationary version of the classical Reed–Frost model is constructed that allows us to incorporate, in particular, random levels of resistance to infection in the susceptibles. Then, a randomized version of this nonstationary model is considered in order to take into account random levels of infectiousness in the infectives. It is shown that, in both cases, the distribution of the final number of infected individuals can be obtained in terms of Abel–Gontcharoff polynomials. The new methodology followed also provides a unified approach to a number of recent works in the literature.

We derive a tight perturbation bound for hidden Markov models. Using this bound, we show that, in many cases, the distribution of a hidden Markov model is considerably more sensitive to perturbations in the emission probabilities than to perturbations in the transition probability matrix and the initial distribution of the underlying Markov chain. Our approach can also be used to assess the sensitivity of other stochastic models, such as mixture processes and semi-Markov processes.

We consider two versions of a simple evolutionary algorithm (EA) model for protein folding at zero temperature, namely the (1 + 1)-EA on the LeadingOnes problem. In this schematic model, the structure of the protein, which is encoded as a bit-string of length n, is evolved to its native conformation through a stochastic pathway of sequential contact bindings. We study the asymptotic behavior of the hitting time, in the mean case scenario, under two different mutations: the one-flip, which flips a unique bit chosen uniformly at random in the bit-string, and the Bernoulli-flip, which flips each bit in the bit-string independently with probability c/n, for some c ∈ ℝ+ (0 ≤ c ≤ n). For each algorithm, we prove a law of large numbers, a central limit theorem, and compare the performance of the two models.

In this paper, we introduce the minimum dynamic discrimination information (MDDI) approach to probability modeling. The MDDI model relative to a given distribution G is that which has least Kullback-Leibler information discrepancy relative to G, among all distributions satisfying some information constraints given in terms of residual moment inequalities, residual moment growth inequalities, or hazard rate growth inequalities. Our results lead to MDDI characterizations of many well-known lifetime models and to the development of some new models. Dynamic information constraints that characterize these models are tabulated. A result for characterizing distributions based on dynamic Rényi information divergence is also given.

As proposed by Irle and Gani in 2001, a process X is said to be slower in level crossing than a process Y if it takes X stochastically longer to exceed any given level than it does Y. In this paper, we extend a result of Irle (2003), relative to the level crossing ordering of uniformizable skip-free-to-the-right continuous-time Markov chains, to derive a new set of sufficient conditions for the level crossing ordering of these processes. We apply our findings to birth-death processes with and without catastrophes, and M/M/s/c systems.

In this paper, we propose a new urn model. A single urn contains b black balls and w white balls. For each observation, we randomly draw m balls and note their colors, say k black balls and m − k white balls. We return the drawn balls to the urn with an additional ck black balls and c(m − k) white balls. We repeat this procedure n times and denote by Xn the fraction of black balls after the nth draw. To investigate the asymptotic properties of Xn, we first perform some computational studies. We then show that {Xn} forms a martingale, which converges almost surely to a random variable X. The distribution of X is then shown to be absolutely continuous.

We study portfolio optimization problems in which the drift rate of the stock is Markov modulated and the driving factors cannot be observed by the investor. Using results from filter theory, we reduce this problem to one with complete observation. In the cases of logarithmic and power utility, we solve the problem explicitly with the help of stochastic control methods. It turns out that the value function is a classical solution of the corresponding Hamilton-Jacobi-Bellman equation. As a special case, we investigate the so-called Bayesian case, i.e. where the drift rate is unknown but does not change over time. In this case, we prove a number of interesting properties of the optimal portfolio strategy. In particular, using the likelihood-ratio ordering, we can compare the optimal investment in the case of observable drift rate to that in the case of unobservable drift rate. Thus, we also obtain the sign of the drift risk.