It is well known that various characteristics in risk and queuing processes can be formulated as Markov renewal functions, which are determined by Markov renewal equations. However, those functions have not been utilized as they are expected. In this article, we show that they are useful for studying asymptotic decay in risk and queuing processes under a Markovian environment. In particular, a matrix version of the Cramér–Lundberg approximation is obtained for the risk process. The corresponding result for the MAP/G/1 queue is presented as well. Emphasis is placed on a straightforward derivation using the Markov renewal structure.

We consider a modulated fluid system with a finite state-space Markov chain Jt as modulating process and general state-dependent net input rates. We derive differential equations for the transient and the stationary distribution of (Wt, Jt), where Wt is the content process, and the corresponding Laplace transforms with respect to time. Moreover, we study the level hitting times of Wt. Our results lead to explicit formulas in the case of two modulating states.

We use a Poisson imbedding technique to investigate the possibilities of generalizing some pathwise and stochastic monotonicity results from the M/M/k queues to systems with monotone failure rate service time distribution. A dichotomy between the decreasing failure rate (DFR) and the increasing failure rate (IFR) cases is revealed: In the DFR case, we achieve that the number of customers in the systems is stochastically increasing if it is idle at time 0, whereas in the IFR case, there is an alternating character that renders a bound between two identical systems that have different initial conditions. We also explore how our methods work in a comparison between systems with different numbers of stations but the same maximal capacity.

In this article we characterize l∞-spherical density functions by means of epoch times of nonhomogeneous pure birth processes. Some further properties of l∞-spherical densities, such as Schur-concavity, positive dependence, and stochastic comparisons, are also given. The relationships of l∞-spherical densities to notions of interest in reliability theory are highlighted.

In this article, we propose a new continuous-time stochastic inventory model with deterioration and stock-dependent demand items. We then formulate the problem of finding the optimal impulse control schedule that minimizes the total expected return over an infinite horizon, as a quasivariational inequality (QVI) problem. The QVI is shown to lead to an (s, S) policy, where s and S are determined uniquely as a solution of some algebraic equations.

We consider a reflected independent superposition of a Brownian motion and a compound Poisson process with positive and negative jumps, which can be interpreted as a model for the content process of a storage system with different types of customers under heavy traffic. The distributions of the duration of a busy cycle and the maximum content during a cycle are determined in closed form.

We consider the problem of dynamic allocation of a single server with batch processing capability to a set of parallel queues. Jobs from different classes cannot be processed together in the same batch. The arrival processes are mutually independent Poisson flows with equal rates. Batches have independent and identically distributed exponentially distributed service times, independent of the batch size and the arrival processes. It is shown that for the case of infinite buffers, allocating the server to the longest queue, stochastically maximizes the aggregate throughput of the system. For the case of equal-size finite buffers the same policy stochastically minimizes the loss of jobs due to buffer overflows. Finally, for the case of unequal-size buffers, a threshold-type policy is identified through an extensive simulation study and shown to consistently outperform other conventional policies. The good performance of the proposed threshold policy is confirmed in the heavy-traffic regime using a fluid model.

We propose a particular class of transition probability matrices for discrete-time Markov chains with a closed form to compute the stationary distribution. The stochastic monotonicity properties of this class are established. We give algorithms to construct monotone, bounding matrices belonging to the proposed class for the variability orders. The accuracy of bounds with respect to the underlying matrix structure is discussed through an example.

In this article, we develop methods for estimating the expected time to the first loss in an Erlang loss system. We are primarily interested in estimating this quantity under light traffic conditions. We propose and compare three simulation techniques as well as two Markov chain approximations. We show that the Markov chain approximations proposed by us are asymptotically exact when the load offered to the system goes to zero. The article also serves to highlight the fact that efficient estimation of transient quantities of stochastic systems often requires the use of techniques that combine analytical results with simulation.

Two discrete time risk models under rates of interest are introduced. Ruin probabilities in the two risk models are discussed. Stochastic inequalities for the ruin probabilities are derived by martingales and renewal recursive techniques. The inequalities can be used to evaluate the ruin probabilities as upper bounds. Numerical illustrations for these results are given.

We discuss the asymptotic behavior of time-inhomogeneous Metropolis chains for solving constrained sampling and optimization problems. In addition to the usual inverse temperature schedule (βn)n∈[hollow N]*, the type of Markov processes under consideration is controlled by a divergent sequence (θn)n∈[hollow N]* of parameters acting as Lagrange multipliers. The associated transition probability matrices (Pβn,θn)n∈[hollow N]* are defined by Pβ,θ = q(x, y)exp(−β(Wθ(y) − Wθ(x))+) for all pairs (x, y) of distinct elements of a finite set Ω, where q is an irreducible and reversible Markov kernel and the energy function Wθ is of the form Wθ = U + θV for some functions U,V : Ω → [hollow R]. Our approach, which is based on a comparison of the distribution of the chain at time n with the invariant measure of Pβn,θn, requires the computation of an upper bound for the second largest eigenvalue in absolute value of Pβn,θn. We extend the geometric bounds derived by Ingrassia and we give new sufficient conditions on the control sequences for the algorithm to simulate a Gibbs distribution with energy U on the constrained set [Ω with tilde above] = {x ∈ Ω : V(x) = minz∈ΩV(z)} and to minimize U over [Ω with tilde above].

For a two-node tandem fluid model with gradual input, we compute the joint steady-state buffer-content distribution. Our proof exploits martingale methods developed by Kella and Whitt. For the case of finite buffers, we use an insightful sample-path argument to extend an earlier proportionality result of Zwart to the network case.

We study systems of parallel queues with finite buffers, a single server with random connectivity to each queue, and arriving job flows with random or class-dependent accessibility to the queues. Only currently connected queues may receive (preemptive) service at any given time, whereas an arriving job can only join one of its accessible queues. Using the coupling method, we study three key models, progressively building from simpler to more complicated structures.

In the first model, there are only random server connectivities. It is shown that allocating the server to the Connected queue with the Fewest Empty Spaces (C-FES) stochastically minimizes the number of lost jobs due to buffer overflows, under conditions of independence and symmetry.

In the second model, we additionally consider random accessibility of queues by arriving jobs. It is shown that allocating the server to the C-FES and routing each arriving job to the currently Accessible queue with the Most Empty Spaces (C-FES/A-MES) minimizes the loss flow stochastically, under similar assumptions.

In the third model (addressing a target application), we consider multiple classes of arriving job flows, each allowed access to a deterministic subset of the queues. Under analogous assumptions, it is again shown that the C-FES/A-MES policy minimizes the loss flow stochastically.

The random connectivity/accessibility aspect enhances significantly the structure and application scope of the classical parallel queuing model. On the other hand, it introduces essential additional dynamics and considerable complications. It is interesting that a simple policy like FES/MES, known to be optimal for the classical model, extends to the C-FES/A-MES in our case.

We consider a single-server queue that is initially empty and operates under the first-in–first-out service discipline. In this system, delays (waiting times in queue) experienced by subsequent arriving customers form a transient process. We investigate its transient behavior by constructing a sample-path coupling of the transient and a general (delayed) processes. From the coupling, we obtain an identity that relates the sample paths of these two processes. This identity helps us to better understand the queue's approach to the stationary limit and to derive upper and lower bounds on the expected transient delay. In addition, we use a Brownian-motion model to approximate the identity. This produces an approximation of the expected transient delay. The approximation turns out to be identical to the corresponding first moment of a reflected Brownian motion. Thus, it is easy to compute and its accuracy is supported by numerical experiments.

We consider the optimal control of two parallel servers in a two-stage tandem queuing system with two flexible servers. New jobs arrive at station 1, after which a series of two operations must be performed before they leave the system. Holding costs are incurred at rate h1 per unit time for each job at station 1 and at rate h2 per unit time for each job at station 2.

The system is considered under two scenarios; the collaborative case and the noncollaborative case. In the prior, the servers can collaborate to work on the same job, whereas in the latter, each server can work on a unique job although they can work on separate jobs at the same station. We provide simple conditions under which it is optimal to allocate both servers to station 1 or 2 in the collaborative case. In the noncollaborative case, we show that the same condition as in the collaborative case guarantees the existence of an optimal policy that is exhaustive at station 1. However, the condition for exhaustive service at station 2 to be optimal does not carry over. This case is examined via a numerical study.

We consider two model variants of a production-inventory system. The system is characterized by a producing machine which is susceptible to failure following which it must be repaired to make it operative again. The machine's production can also be stopped deliberately because of stocking capacity limitations. During ON periods the input into the buffer is continuous and uniform (until a threshold is reached), whereas during OFF periods the output from the buffer is a compound Poisson process. We are interested in computing the equilibrium content level process under the assumption that full backlogging is allowed. In the first model, variant OFF periods are independent of the demand process, and in the second variant, they are determined and controlled in accordance with a certain level crossing stopping rule.

We consider a system with heterogeneous unreliable components that requires only one component to be turned on in order for it to operate. Repair workers may have different skills and may be unavailable for random periods of time. The problem is to determine a usage and repair policy to maximize system availability. We give conditions under which the optimal usage policy is to always use, or turn on, the component with the shortest repair time, and the optimal repair policy is to always repair the most reliable component (with the smallest failure rate). We fully characterize the optimal policy when there are only two components. Our system is equivalent to a closed system with multiple single-server queues, where the objective is to minimize server idle time at one of the queues.

We consider a queue fed by a large number, say n, on–off sources with generally distributed on- and off-times. The queueing resources are scaled by n: The buffer is B ≡ nb and the link rate is C ≡ nc. The model is versatile. It allows one to model both long-range-dependent traffic (by using heavy-tailed on-periods) and short-range-dependent traffic (by using light-tailed on-periods). A crucial performance metric in this model is the steady state buffer overflow probability.

This probability decays exponentially in n. Therefore, if n grows large, naive simulation is too time-consuming and fast simulation techniques have to be used. Due to the exponential decay (in n), importance sampling with an exponential change of measure goes through, irrespective of the on-times being heavy or light tailed. An asymptotically optimal change of measure is found by using large deviations arguments. Notably, the change of measure is not constant during the simulation run, which is different from many other studies (usually relying on large buffer asymptotics).

Numerical examples show that our procedure improves considerably over naive simulation. We present accelerations, we discuss the influence of the shape of the distributions on the overflow probability, and we describe the limitations of our technique.

This article studies the geometric convergence rate of a discrete renewal sequence with decreasing hazard rate or, more generally, new worse than used lifetimes. Several variants of these structural orderings are considered. The results are derived from power series methods; roots of generating functions are the prominent issue. Optimality of the rates are considered. Examples demonstrating the utility of the results, as well as applications to Markov chains, are presented.

In this article, we establish stochastic comparisons between normalized spacings of generalized order statistics. These comparisons allow us to extend and unify some results obtained by other authors for ordinary order statistics and record values. Furthermore, we can compare spacings of different models (i.e., between ordinary order statistics and sequential order statistics, record values and Pfeifer's record values, and so forth).