This paper is concerned with the analysis of Markov decision processes in which a natural form of termination ensures that the expected future costs are bounded, at least under some policies. Whereas most previous analyses have restricted attention to the case where the set of states is finite, this paper analyses the case where the set of states is not necessarily finite or even countable. It is shown that all the existence, uniqueness, and convergence results of the finite-state case hold when the set of states is a general Borel space, provided we make the additional assumption that the optimal value function is bounded below. We give a sufficient condition for the optimal value function to be bounded below which holds, in particular, if the set of states is countable.

We show that the maximal expected utility satisfies a monotone continuity property with respect to increasing information. Let be a sequence of increasing filtrations converging to , and let un(x) and u∞(x) be the maximal expected utilities when investing in a financial market according to strategies adapted to and , respectively. We give sufficient conditions for the convergence un(x) → u∞(x) as n → ∞. We provide examples in which convergence does not hold. Then we consider the respective utility-based prices, πn and π∞, of contingent claims under (Gtn) and (Gt∞). We analyse to what extent πn → π∞ as n → ∞.

In an incomplete financial market in which the dynamics of the asset prices is driven by a d-dimensional continuous semimartingale X, we consider the problem of pricing European contingent claims embedded in a power utility framework. This problem reduces to identifying the p-optimal martingale measure, which can be given in terms of the solution to a semimartingale backward equation. We use this characterization to examine two extreme cases. In particular, we find a necessary and sufficient condition, written in terms of the mean-variance trade-off, for the p-optimal martingale measure to coincide with the minimal martingale measure. Moreover, if and only if an exponential function of the mean-variance trade-off is a martingale strongly orthogonal to the asset price process, the p-optimal martingale measure can be simply expressed in terms of a Doléans-Dade exponential involving X.

In this paper we propose a new method of determining the stability of queueing systems. We attain it using the absorbing process and introduce the untraceable events method to show the existence of the absorbing process. The advantage of our method is that we are able to discuss the stability of various variables for both discrete and continuous parameters in a general framework with nonstationary input. An untraceable event has the property that the state loses the memory of its origin. In a concrete model, we use the boundedness of the state at an epoch in time with respect to the initial condition and choose the form of the untraceable event corresponding to the input distribution.

We consider a family of long-range percolation models (Gp)p>0 on ℤd that allow dependence between edges and have the following connectivity properties for p ∈ (1/d, ∞): (i) the degree distribution of vertices in Gp has a power-law distribution; (ii) the graph distance between points x and y is bounded by a multiple of logpdlogpd|x - y| with probability 1 - o(1); and (iii) an adversary can delete a relatively small number of nodes from Gp(ℤd ∩ [0, n]d), resulting in two large, disconnected subgraphs.

In a recent paper, Kleinberg (2000) considered a small-world network model consisting of a d-dimensional lattice augmented with shortcuts. The probability of a shortcut being present between two points decays as a power, r-α, of the distance, r, between them. Kleinberg showed that greedy routeing is efficient if α = d and that there is no efficient decentralised routeing algorithm if α ≠ d. The results were extended to a continuum model by Franceschetti and Meester (2003). In our work, we extend the result to more realistic models constructed from a Poisson point process wherein each point is connected to all its neighbours within some fixed radius, and possesses random shortcuts to more distant nodes as described above.

In this paper we consider stochastic recursive equations of sum type, , and of max type, , where Ai, bi, and b are random, (Xi) are independent, identically distributed copies of X, and denotes equality in distribution. Equations of these types typically characterize limits in the probabilistic analysis of algorithms, in combinatorial optimization problems, and in many other problems having a recursive structure. We develop some new contraction properties of minimal Ls-metrics which allow us to establish general existence and uniqueness results for solutions without imposing any moment conditions. As an application we obtain a one-to-one relationship between the set of solutions to the homogeneous equation and the set of solutions to the inhomogeneous equation, for sum- and max-type equations. We also give a stochastic interpretation of a recent transfer principle of Rösler from nonnegative solutions of sum type to those of max type, by means of random scaled Weibull distributions.

We characterize the (continuous) majorization of integrable functions introduced by Hardy, Littlewood, and Pólya in terms of the (discrete) majorization of finite-dimensional vectors, introduced by the same authors. The most interesting version of this result is the characterization of the (increasing) convex order for integrable random variables in terms of majorization of vectors of expected order statistics. Such a result includes, as particular cases, previous results by Barlow and Proschan and by Alzaid and Proschan, and, in a sense, completes the picture of known results on order statistics. Applications to other stochastic orders are also briefly considered.

We consider risk processes with reinsurance. A general family of reinsurance contracts is allowed, including proportional and excess-of-loss policies. Claim occurrence is regulated by a classical compound Poisson process or by a Markov-modulated compound Poisson process. We provide some large deviation results concerning these two risk processes in the small-claim case. Finally, we derive the so-called Lundberg estimate for the ruin probabilities and present a numerical example.

A sequence of objects with independent, identically distributed qualities is presented to a selector who must choose two on-line, i.e. without anticipation or recall. The selector's aim is to obtain a satisfactory pair as quickly as possible. Two versions of the problem are considered, and optimal selection rules are derived and compared. An investigation is also made of a heuristic rule suitable for a selector who has no prior knowledge of the nature of the object sequence.

We study exponential families within the class of counting processes and show that a mixed Poisson process belongs to an exponential family if and only if it is either a Poisson process or has a gamma structure distribution. This property can be expressed via exponential martingales.

Consider N players, respectively owning x1, x2, …, xN monetary units, who play a sequence of games, winning from and losing to each other integer amounts according to fixed rules. The sequence stops as soon as (at least) one player is ruined. We are interested in the ruin process of these N players, i.e. in the probability that a given player is ruined first, and also in the expected ruin time. This problem is called the N-player ruin problem. In this paper, the problem is set up as a multivariate absorbing Markov chain with an absorbing state corresponding to the ruin of each player. This is then discussed in the context of phase-type distributions where each phase is represented by a vector of size N and the distribution has as many absorbing points as there are ruin events. We use this modified phase-type distribution to obtain an explicit solution to the N-player problem. We define a partition of the set of transient states into different levels, and on it give an extension of the folding algorithm (see Ye and Li (1994)). This provides an efficient computational procedure for calculating some of the key measures.

In this paper we obtain some ergodic properties and ergodic decompositions of a continuous-time, Borel right Markov process taking values in a locally compact and separable metric space. Initially, we assume that an invariant probability measure (IPM) μ exists for the process and, without making any further assumptions on the transition kernel, obtain some characterization results for the convergence of the expected occupation measure to a limit kernel. Under the same assumption, we present the so-called Yosida decomposition. Next, instead of assuming the existence of an IPM, we assume that the Markov process satisfies a certain condition, named the T'-condition. Under this condition it is shown that the Foster-Lyapunov criterion is necessary and sufficient for the existence of an IPM and that the process admits a Doeblin decomposition. Furthermore, it is shown that in this case the set of ergodic probability measures is countable and that every probability measure for the Markov process is nonsingular with respect to the transition kernel.

The northeast model is a spin system on the two-dimensional integer lattice that evolves according to the following rule: whenever a site's southerly and westerly nearest neighbors have spin 1, it may reset its own spin by tossing a p-coin; at all other times, its spin remains frozen. It is proved that the northeast model has a phase transition at pc = 1 - βc, where βc is the critical parameter for oriented percolation. For p < pc, the trivial measure, δ0, that puts mass one on the configuration with all spins set at 0 is the unique ergodic, translation-invariant, stationary measure. For p ≥ pc, the product Bernoulli-p measure on configuration space is the unique nontrivial, ergodic, translation-invariant, stationary measure for the system, and it is mixing. For p > ⅔, it is shown that there is exponential decay of correlations.

We compare dependence in stochastically monotone Markov processes with partially ordered Polish state spaces using the concordance and supermodular orders. We show necessary and sufficient conditions for the concordance order to hold both in terms of the one-step transition probabilities for discrete-time processes and in terms of the corresponding infinitesimal generators for continuous-time processes. We give examples showing that a stochastic monotonicity assumption is not necessary for such orderings. We indicate relations between dependence orderings and, variously, the asymptotic variance-reduction effect in Monte Carlo Markov chains, Cheeger constants, and positive dependence for Markov processes.

Nonexponential asymptotics for solutions of two specific defective renewal equations are obtained. These include the special cases of asymptotics for a compound geometric distribution and the convolution of a compound geometric distribution with a distribution function. As applications of these results, we study the asymptotic behavior of the demographic birth rate of females, the perpetual put option in mathematics of finance, and the renewal function for terminating renewal processes.

In this paper we present a general framework for the modelling of the process of corrective and condition-based preventive maintenance actions for complex repairable systems. A new class of models is proposed, the generalized virtual age models. On the one hand, these models generalize Kijima's virtual age models to the case where both preventive and corrective maintenances are present. On the other hand, they generalize the usual competing risks models to imperfect maintenance actions which do not renew the system. A generalized virtual age model is defined by both a sequence of effective ages which characterizes the effects of both types of maintenance according to a classical virtual age model, and a usual competing risks model which characterizes the dependency between the two types of maintenance. Several particular cases of the general model are derived.

We study the durations of the avalanches in the maximal avalanche decomposition of the Bak-Sneppen evolution model. We show that all the avalanches in this maximal decomposition have infinite expectation, but only ‘barely’, in the sense that if we made the appropriate threshold a tiny amount smaller (in a certain sense), then the avalanches would have finite expectation. The first of these results is somewhat surprising, since simulations suggest finite expectations.

In the present article we derive a new discrete multivariate distribution using a limiting argument that is essentially the same as the law of small numbers. The distribution derived belongs to an exponential family, and randomly partitions positive integers. The facts shown about the distribution are useful in many fields of application involved with count data. The derivation parallels that of the Ewens distribution from the gamma distribution, and the new distribution is produced from the inverse Gaussian distribution. The method employed is regarded as the discretization of an infinitely divisible distribution over nonnegative real numbers.

We provide bounds for perpetual American option prices in a jump diffusion model in terms of American option prices in the standard Black–Scholes model. We also investigate the dependence of the bounds on different parameters of the model.