We provide exact computations for the drift of random walks in dependent random environments, including k-dependent and moving average environments. We show how the drift can be characterized and evaluated using Perron–Frobenius theory. Comparing random walks in various dependent environments, we demonstrate that their drifts can exhibit interesting behavior that depends significantly on the dependency structure of the random environment.

In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.

We propose a novel theoretical characterization of the optimal 'Gittins index' policy in multi-armed bandit problems with non-Gaussian, infinitely divisible reward distributions. We first construct a continuous-time, conditional Lévy process which probabilistically interpolates the sequence of discrete-time rewards. When the rewards are Gaussian, this approach enables an easy connection to the convenient time-change properties of a Brownian motion. Although no such device is available in general for the non-Gaussian case, we use optimal stopping theory to characterize the value of the optimal policy as the solution to a free-boundary partial integro-differential equation (PIDE). We provide the free-boundary PIDE in explicit form under the specific settings of exponential and Poisson rewards. We also prove continuity and monotonicity properties of the Gittins index in these two problems, and discuss how the PIDE can be solved numerically to find the optimal index value of a given belief state.

We study the asymptotic behavior of a new particle filter approach for the estimation of hidden Markov models. In particular, we develop an algorithm where the latent-state sequence is segmented into multiple shorter portions, with an estimation technique based upon a separate particle filter in each portion. The partitioning facilitates the use of parallel processing, which reduces the wall-clock computational time. Based upon this approach, we introduce new estimators of the latent states and likelihood which have similar or better variance properties compared to estimators derived from standard particle filters. We show that the likelihood function estimator is unbiased, and show asymptotic normality of the underlying estimators.

We consider a class of multi-type particle systems whose structure is similar to that of a contact process and show that additivity is equivalent to the existence of a dual process, extending a result of Harris. We prove a necessary and sufficient condition for the model to preserve positive correlations. We then show that complete convergence on Zd holds for a large subclass of models including the two-stage contact process and a household model, and give examples.

We consider a threshold epidemic model on a clustered random graph model obtained from local transformations in an alternating branching process that approximates a bipartite graph. In other words, our epidemic model is such that an individual becomes infected as soon as the proportion of his/her infected neighbors exceeds the threshold q of the epidemic. In our random graph model, each individual can belong to several communities. The distributions for the community sizes and the number of communities an individual belongs to are arbitrary. We consider the case where the epidemic starts from a single individual, and we prove a phase transition (when the parameter q of the model varies) for the appearance of a cascade, i.e. when the epidemic can be propagated to an infinite part of the population. More precisely, we show that our epidemic is entirely described by a multi-type (and alternating) branching process, and then we apply Sevastyanov's theorem about the phase transition of multi-type Galton-Watson branching processes. In addition, we compute the entries of the mean progeny matrix corresponding to the epidemic. The phase transition for the contagion is given in terms of the largest eigenvalue of this matrix.

Density dependent Markov population processes in large populations of size N were shown by Kurtz (1970), (1971) to be well approximated over finite time intervals by the solution of the differential equations that describe their average drift, and to exhibit stochastic fluctuations about this deterministic solution on the scale √N that can be approximated by a diffusion process. Here, motivated by an example from evolutionary biology, we are concerned with describing how such a process leaves an absorbing boundary. Initially, one or more of the populations is of size much smaller than N, and the length of time taken until all populations have sizes comparable to N then becomes infinite as N → ∞. Under suitable assumptions, we show that in the early stages of development, up to the time when all populations have sizes at least N1-α for 1/3 < α < 1, the process can be accurately approximated in total variation by a Markov branching process. Thereafter, it is well approximated by the deterministic solution starting from the original initial point, but with a random time delay. Analogous behaviour is also established for a Markov process approaching an equilibrium on a boundary, where one or more of the populations become extinct.

This paper concerns the asymptotic behavior of a random variable Wλ resulting from the summation of the functionals of a Gibbsian spatial point process over windows Qλ ↑ ℝd. We establish conditions ensuring that Wλ has volume order fluctuations, i.e. they coincide with the fluctuations of functionals of Poisson spatial point processes. We combine this result with Stein's method to deduce rates of a normal approximation for Wλ as λ → ∞. Our general results establish variance asymptotics and central limit theorems for statistics of random geometric and related Euclidean graphs on Gibbsian input. We also establish a similar limit theory for claim sizes of insurance models with Gibbsian input, the number of maximal points of a Gibbsian sample, and the size of spatial birth-growth models with Gibbsian input.

We develop diffusion approximations for parallel-queueing systems with the randomized longest-queue-first scheduling (LQF) algorithm by establishing new mean-field limit theorems as the number of buffers n → ∞. We achieve this by allowing the number of sampled buffers d = d(n) to depend on the number of buffers n, which yields an asymptotic 'decoupling' of the queue length processes. We show through simulation experiments that the resulting approximation is accurate even for moderate values of n and d(n). To the best of the authors' knowledge, this is the first derivation of diffusion approximations for a queueing system in the large-buffer mean-field regime. Another noteworthy feature of our scaling idea is that the randomized LQF algorithm emulates the LQF algorithm, yet is computationally more attractive. The analysis of the system performance as a function of d(n) is facilitated by the multi-scale nature in our limit theorems: the various processes we study have different space scalings. This allows us to show the trade-off between performance and complexity of the randomized LQF scheduling algorithm.

The Doss-Sussmann (DS) approach is used for uniform simulation of the Cox-Ingersoll-Ross (CIR) process. The DS formalism allows us to express trajectories of the CIR process through solutions of some ordinary differential equation (ODE) depending on realizations of a Wiener process involved. By simulating the first-passage times of the increments of the Wiener process to the boundary of an interval and solving the ODE, we uniformly approximate the trajectories of the CIR process. In this respect special attention is payed to simulation of trajectories near 0. From a conceptual point of view the proposed method gives a better quality of approximation (from a pathwise point of view) than standard, even exact, simulation of the stochastic differential equation at some deterministic time grid.

Fractional Lévy processes generalize fractional Brownian motion in a natural way. We go a step further and extend the usual fractional Riemann-Liouville kernel to a regularly varying function. We call the resulting stochastic processes generalized fractional Lévy processes (GFLPs) and show that they may have short or long memory increments and that their sample paths may have jumps or not. Moreover, we define stochastic integrals with respect to a GFLP and investigate their second-order structure and sample path properties. A specific example is the Ornstein-Uhlenbeck process driven by a time-scaled GFLP. We prove a functional central limit theorem for such scaled processes with a fractional Ornstein-Uhlenbeck process as a limit process. This approximation applies to a wide class of stochastic volatility models, which include models where possibly neither the data nor the latent volatility process are semimartingales.

In this paper we focus on the finite-horizon optimality for denumerable continuous-time Markov decision processes, in which the transition and reward/cost rates are allowed to be unbounded, and the optimality is over the class of all randomized history-dependent policies. Under mild reasonable conditions, we first establish the existence of a solution to the finite-horizon optimality equation by designing a technique of approximations from the bounded transition rates to unbounded ones. Then we prove the existence of ε (≥ 0)-optimal Markov policies and verify that the value function is the unique solution to the optimality equation by establishing the analog of the Itô-Dynkin formula. Finally, we provide an example in which the transition rates and the value function are all unbounded and, thus, obtain solutions to some of the unsolved problems by Yushkevich (1978).

We derive Laplace transform identities for the volume content of random stopping sets based on Poisson point processes. Our results are based on anticipating Girsanov identities for Poisson point processes under a cyclic vanishing condition for a finite difference gradient. This approach does not require classical assumptions based on set-indexed martingales and the (partial) ordering of index sets. The examples treated focus on stopping sets in finite volume, and include the random missed volume of Poisson convex hulls.

In this paper we describe a perfect simulation algorithm for the stable M/G/c queue. Sigman (2011) showed how to build a dominated coupling-from-the-past algorithm for perfect simulation of the super-stable M/G/c queue operating under first-come-first-served discipline. Sigman's method used a dominating process provided by the corresponding M/G/1 queue (using Wolff's sample path monotonicity, which applies when service durations are coupled in order of initiation of service). The method exploited the fact that the workload process for the M/G/1 queue remains the same under different queueing disciplines, in particular under the processor sharing discipline, for which a dynamic reversibility property holds. We generalise Sigman's construction to the stable case by comparing the M/G/c queue to a copy run under random assignment. This allows us to produce a naïve perfect simulation algorithm based on running the dominating process back to the time it first empties. We also construct a more efficient algorithm that uses sandwiching by lower and upper processes constructed as coupled M/G/c queues started respectively from the empty state and the state of the M/G/c queue under random assignment. A careful analysis shows that appropriate ordering relationships can still be maintained, so long as service durations continue to be coupled in order of initiation of service. We summarise statistical checks of simulation output, and demonstrate that the mean run-time is finite so long as the second moment of the service duration distribution is finite.

We study optimal double stopping problems driven by a Brownian bridge. The objective is to maximize the expected spread between the payoffs achieved at the two stopping times. We study several cases where the solutions can be solved explicitly by strategies of a threshold type.

We consider dynamic versions of the mutual information of lifetime distributions, with a focus on past lifetimes, residual lifetimes, and mixed lifetimes evaluated at different instants. This allows us to study multicomponent systems, by measuring the dependence in conditional lifetimes of two components having possibly different ages. We provide some bounds, and investigate the mutual information of residual lifetimes within the time-transformed exponential model (under both the assumptions of unbounded and truncated lifetimes). Moreover, with reference to the order statistics of a random sample, we evaluate explicitly the mutual information between the minimum and the maximum, conditional on inspection at different times, and show that it is distribution-free in a special case. Finally, we develop a copula-based approach aiming to express the dynamic mutual information for past and residual bivariate lifetimes in an alternative way.

We study cyclic polling models with exhaustive service at each queue under a variety of non-FCFS (first-come-first-served) local service orders, namely last-come-first-served with and without preemption, random-order-of-service, processor sharing, the multi-class priority scheduling with and without preemption, shortest-job-first, and the shortest remaining processing time policy. For each of these policies, we first express the waiting-time distributions in terms of intervisit-time distributions. Next, we use these expressions to derive the asymptotic waiting-time distributions under heavy-traffic assumptions, i.e. when the system tends to saturate. The results show that in all cases the asymptotic waiting-time distribution at queue i is fully characterized and of the form Γ Θi, with Γ and Θi independent, and where Γ is gamma distributed with known parameters (and the same for all scheduling policies). We derive the distribution of the random variable Θi which explicitly expresses the impact of the local service order on the asymptotic waiting-time distribution. The results provide new fundamental insight into the impact of the local scheduling policy on the performance of a general class of polling models. The asymptotic results suggest simple closed-form approximations for the complete waiting-time distributions for stable systems with arbitrary load values.