A forwards induction policy is a type of greedy algorithm for Markov decision processes. It is straightforward to implement and is optimal for a large class of models, especially in stochastic resource allocation. In this paper we consider a model for the optimal allocation of resources in pre-clinical pharmaceutical research. We show that although they are not always strictly optimal, forwards induction policies perform well.

For a given bivariate Lévy process (Ut, Lt)t≥0, distributional properties of the stationary solutions of the stochastic differential equation dVt = Vt-dUt + dLt are analysed. In particular, the expectation and autocorrelation function are obtained in terms of the process (U, L) and in several cases of interest the tail behavior is described. In the case where U has jumps of size −1, necessary and sufficient conditions for the law of the solutions to be (absolutely) continuous are given.

This paper is concerned with the behaviour of a Lévy process when it crosses over a positive level, u, starting from 0, both as u becomes large and as u becomes small. Our main focus is on the time, τu, it takes the process to transit above the level, and in particular, on the stability of this passage time; thus, essentially, whether or not τu behaves linearly as u ↓ 0 or u → ∞. We also consider the conditional stability of τu when the process drifts to -∞ almost surely. This provides information relevant to quantities associated with the ruin of an insurance risk process, which we analyse under a Cramér condition.

We consider the problem of maximizing the probability of stopping on any of the last m successes in independent Bernoulli trials with random horizon of length N, where m is a predetermined integer. A prior is given for N. It is known that, when N is degenerate, i.e. P{N = n} = 1 for a given n > m, the sum-the-multiplicative-odds theorem gives the solution and shows that the optimal rule is a threshold rule, i.e. it stops on the first success appearing after a given stage. However, when N is nondegenerate, the optimal rule is not necessarily a threshold rule. So our main concern in Section 2 is to give a sufficient condition for the optimal rule to be a threshold rule when N is a bounded random variable such that P{N ≤ n} = 1. Application will be made to the usual (discrete arrival time) secretary problem with a random number N of applicants in Section 3. When N is uniform or curtailed geometric, the optimal rules are shown to be threshold rules and their asymptotic results are obtained. We also examine, as a nonhomogeneous Poisson process model, an intermediate prior that allows N to be uniform or degenerate. In Section 4 we consider a continuous arrival time version of the secretary problem with a random number M of applicants. It is shown that, whatever the distribution of M, we can win with probability greater than or equal to um*, where um* is, as given in (1.4), the asymptotic win probability of the usual secretary problem when N degenerates to n and n → ∞.

We consider a stochastic control model for a queueing system driven by a two-dimensional fractional Brownian motion with Hurst parameter 0 < H < 1. In particular, when H > ½, this model serves to approximate a controlled two-station tandem queueing model with heavy-tailed ON/OFF sources in heavy traffic. We establish the weak convergence results for the distribution of the state process and construct an explicit stationary state process associated with given controls. Based on suitable coupling arguments, we show that each state process couples with its stationary counterpart and we use it to represent the long-run average cost functional in terms of the stationary process. Finally, we establish the existence result of an optimal control, which turns out to be independent of the initial data.

In this paper we consider the scaled limit of a continuous-time random walk (CTRW) based on a Markov chain {Xn, n ≥ 0} and two observables, τ(∙) and V(∙), corresponding to the renewal times and jump sizes. Assuming that these observables belong to the domains of attraction of some stable laws, we give sufficient conditions on the chain that guarantee the existence of the scaled limits for CTRWs. An application of the results to a process that arises in quantum transport theory is provided. The results obtained in this paper generalize earlier results contained in Becker-Kern, Meerschaert and Scheffler (2004) and Meerschaert and Scheffler (2008), and the recent results of Henry and Straka (2011) and Jurlewicz, Kern, Meerschaert and Scheffler (2010), where {Xn, n ≥ 0} is a sequence of independent and identically distributed random variables.

In the so-called lightbulb process, on days r = 1,…,n, out of n lightbulbs, all initially off, exactly r bulbs, selected uniformly and independent of the past, have their status changed from off to on, or vice versa. With X the number of bulbs on at the terminal time n, an even integer, and μ = n/2, σ2 = var(X), we have supz ∈ R | P((X - μ)/σ ≤ z) - P(Z ≤ z) | ≤ nΔ̅0/2σ2 + 1.64n/σ3 + 2/σ, where Z is a standard normal random variable and Δ̅0 = 1/2√n + 1/2n + e−n/2/3 for n ≥ 6, yielding a bound of order O(n−1/2) as n → ∞. A similar, though slightly larger bound, holds for odd n. The results are shown using a version of Stein's method for bounded, monotone size bias couplings. The argument for even n depends on the construction of a variable Xs on the same space as X that has the X-size bias distribution, that is, which satisfies E[Xg(X)] = μE[g(Xs)] for all bounded continuous g, and for which there exists a B ≥ 0, in this case B = 2, such that X ≤ Xs ≤ X + B almost surely. The argument for odd n is similar to that for even n, but one first couples X closely to V, a symmetrized version of X, for which a size bias coupling of V to Vs can proceed as in the even case. In both the even and odd cases, the crucial calculation of the variance of a conditional expectation requires detailed information on the spectral decomposition of the lightbulb chain.

We consider the model of Deijfen, Häggström and Bagley (2004) for competing growth of two infection types in Rd, based on the Richardson model on Zd. Stochastic ball-shaped infection outbursts transmit the infection type of the center to all points of the ball that are not yet infected. Relevant parameters of the model are the initial infection configuration, the (type-dependent) growth rates, and the radius distribution of the infection outbursts. The main question is that of coexistence: Which values of the parameters allow the unbounded growth of both types with positive probability? Deijfen, Häggström and Bagley (2004) conjectured that the initial configuration is basically irrelevant for this question, and gave a proof for this under strong assumptions on the radius distribution, which, e.g. do not include the case of a deterministic radius. Here we give a proof that does not rely on these assumptions. One of the tools to be used is a slight generalization of the model with immune regions and delayed initial infection configurations.

We introduce a new point process, the dynamic contagion process, by generalising the Hawkes process and the Cox process with shot noise intensity. Our process includes both self-excited and externally excited jumps, which could be used to model the dynamic contagion impact from endogenous and exogenous factors of the underlying system. We have systematically analysed the theoretical distributional properties of this new process, based on the piecewise-deterministic Markov process theory developed in Davis (1984), and the extension of the martingale methodology used in Dassios and Jang (2003). The analytic expressions of the Laplace transform of the intensity process and the probability generating function of the point process have been derived. An explicit example of specified jumps with exponential distributions is also given. The object of this study is to produce a general mathematical framework for modelling the dependence structure of arriving events with dynamic contagion, which has the potential to be applicable to a variety of problems in economics, finance, and insurance. We provide an application of this process to credit risk, and a simulation algorithm for further industrial implementation and statistical analysis.

We consider the problem of estimating a latent point process, given the realization of another point process. We establish an expression for the conditional distribution of a latent Poisson point process given the observation process when the transformation from the latent process to the observed process includes displacement, thinning, and augmentation with extra points. Our original analysis is based on an elementary and self-contained random measure theoretic approach. This simplifies and complements previous derivations given in Mahler (2003), and Singh, Vo, Baddeley and Zuyev (2009).

We study optimal control problems for (time-)delayed stochastic differential equations with jumps. We establish sufficient and necessary stochastic maximum principles for an optimal control of such systems. The associated adjoint processes are shown to satisfy a (time-)advanced backward stochastic differential equation (ABSDE). Several results on existence and uniqueness of such ABSDEs are shown. The results are illustrated by an application to optimal consumption from a cash flow with delay.

We consider the following stochastic model for a mobile service scenario. Consider a stationary Poisson process in Rd, with its points radially ordered with respect to the origin (the anchor); if d = 2, the points may correspond to locations of, e.g. restaurants. A user, with a location different from the origin, asks for the location of the first Poisson point and keeps asking for the location of the next Poisson point until the first time that he/she can be completely certain that he/she knows which Poisson point is his/her nearest neighbour. This waiting time is the communication cost, while the inferred privacy region is a random set obtained by an adversary who only knows the anchor and the points received from the server, where the adversary ‘does the best’ to infer the possible locations of the user. Probabilistic results related to the communication cost and the inferred privacy region are established for any dimension d ≥ 1. Furthermore, special results when d = 1 and particularly when d = 2 are derived.

The modeling of random bi-phasic, or porous, media has been, and still is, under active investigation by mathematicians, physicists, and physicians. In this paper we consider a thresholded random process X as a source of the two phases. The intervals when X is in a given phase, named chords, are the subject of interest. We focus on the study of the tails of the chord length distribution functions. In the literature concerned with real data, different types of tail behavior have been reported, among them exponential-like or power-like decay. We look for the link between the dependence structure of the underlying thresholded process X and the rate of decay of the chord length distribution. When the process X is a stationary Gaussian process, we relate the latter to the rate at which the covariance function of X decays at large lags. We show that exponential, or nearly exponential, decay of the tail of the distribution of the chord lengths is very common, perhaps surprisingly so.

Radix Sort is a sorting algorithm based on analyzing digital data. We study the number of swaps made by Radix Select (a one-sided version of Radix Sort) to find an element with a randomly selected rank. This kind of grand average provides a smoothing over all individual distributions for specific fixed-order statistics. We give an exact analysis for the grand mean and an asymptotic analysis for the grand variance, obtained by poissonization, the Mellin transform, and depoissonization. The digital data model considered is the Bernoulli(p). The distributions involved in the swaps experience a phase change between the biased cases (p ≠ ½) and the unbiased case (p = ½). In the biased cases, the grand distribution for the number of swaps (when suitably scaled) converges to that of a perpetuity built from a two-point distribution. The tool for this proof is contraction in the Wasserstein metric space, and identifying the limit as the fixed-point solution of a distributional equation. In the unbiased case the same scaling for the number of swaps gives a limiting constant in probability.

In a fork-join network each incoming job is split into K tasks and the K tasks are simultaneously assigned to K parallel service stations for processing. For the distributions of response times and queue lengths of fork-join networks, no explicit formulae are available. Existing methods provide only analytic approximations for the response time and the queue length distributions. The accuracy of such approximations may be difficult to justify for some complicated fork-join networks. In this paper we propose a perfect simulation method based on coupling from the past to generate exact realisations from the equilibrium of fork-join networks. Using the simulated realisations, Monte Carlo estimates for the distributions of response times and queue lengths of fork-join networks are obtained. Comparisons of Monte Carlo estimates and theoretical approximations are also provided. The efficiency of the sampling algorithm is shown theoretically and via simulation.

Organisms adapt to fluctuating environments by regulating their dynamics, and by adjusting their phenotypes to environmental changes. We model population growth using multitype branching processes in random environments, where the offspring distribution of some organism having trait t ∈ in environment e ∈ ε is given by some (fixed) distribution ϒt,e on ℕ. Then, the phenotypes are attributed using a distribution (strategy) πt,e on the trait space . We look for the optimal strategy πt,e, t ∈ , e ∈ ε, maximizing the net growth rate or Lyapounov exponent, and characterize the set of optimal strategies. This is considered for various models of interest in biology: hereditary versus nonhereditary strategies and strategies involving or not involving a sensing mechanism. Our main results are obtained in the setting of nonhereditary strategies: thanks to a reduction to simple branching processes in a random environment, we derive an exact expression for the net growth rate and a characterization of optimal strategies. We also focus on typical genealogies, that is, we consider the problem of finding the typical lineage of a randomly chosen organism.

For stationary Poisson hyperplane tessellations in d-dimensional Euclidean space and a dimension k ∈ {1, …, d}, we investigate the typical k-face and the weighted typical k-face (weighted by k-dimensional volume), without isotropy assumptions on the tessellation. The case k = d concerns the previously studied typical cell and zero cell, respectively. For k < d, we first find the conditional distribution of the typical k-face or weighted typical k-face, given its direction. Then we investigate how the shapes of the faces are influenced by assumptions of different types: either via containment of convex bodies of given volume (including a new result for k = d), or, for weighted typical k-faces, in the spirit of D. G. Kendall's asymptotic problem, suitably generalized. In all these results on typical or weighted typical k-faces with given direction space L, the Blaschke body of the section process of the underlying hyperplane process with L plays a crucial role.

We introduce a new 1-dependent percolation model to describe and analyze the spread of an epidemic on a general directed and locally finite graph. We assign a two-dimensional random weight vector to each vertex of the graph in such a way that the weights of different vertices are independent and identically distributed, but the two entries of the vector assigned to a vertex need not be independent. The probability for an edge to be open depends on the weights of its end vertices, but, conditionally on the weights, the states of the edges are independent of each other. In an epidemiological setting, the vertices of a graph represent the individuals in a (social) network and the edges represent the connections in the network. The weights assigned to an individual denote its (random) infectivity and susceptibility, respectively. We show that one can bound the percolation probability and the expected size of the cluster of vertices that can be reached by an open path starting at a given vertex from above by the corresponding quantities for independent bond percolation with a certain density; this generalizes a result of Kuulasmaa (1982). Many models in the literature are special cases of our general model.