The stability properties of the bandwidth allocation algorithm First Fit are analyzed for some distributions on the sizes of the requests. Fluid limits are used to get the ergodicity results. When there are two possible sizes, the description of the transient behavior involves a finite Markov chain on the exit states of a transient Markov chain on a countable state space. The explicit expression of this exit matrix is given.

In this article, we identify conditions under which the epoch times and the interepoch intervals of a nonhomogeneous Poisson process have logconcave densities. The results are extended to relevation counting processes. We also study discrete-time counting processes and find conditions under which the epoch times and the interepoch intervals of these discrete-time processes have logconcave discrete probability densities. The results are interpreted in terms of minimal repair and record values. Several examples illustrate the theory.

Some class properties of the used better (worse) than aged [UBA (UWA)] and the used better (worse) than aged in expectation [UBAE (UWAE)] classes of lifetime distributions are considered. Relationships with the decreasing (increasing) mean residual lifetime [DMRL (IMRL)] class and the decreasing (increasing) variance residual lifetime [DVRL (IVRL)] class are established. Discrete UBA and UWA distributions are introduced and studied. Characterizations of UBA and UWA distributions are derived by using discrete aging properties of mixed Poisson distributions. Applications of these results to queueing theory and ruin are then considered. In particular, preservation of UBA (UWA) and UBAE (UWAE) under a transform of life distributions is given.

In this article, we formalize the framework for the supplementary variable technique in stochastic models. Specifically, we show that the use of remaining or elapsed times (or any metric) as supplementary variables leads to the notion of forward or backward Chapman–Kolmogorov equations, respectively. We further show that for a class of queueing systems, using remaining time as the supplementary variable makes analysis simpler.

This paper investigates blocking probabilities obtained from multidimensional truncated Poisson distributions. For blocking probabilities typically arising in layered cellular mobile communications networks, the large deviations results of Gazdzicki et al. [9] are extended to state spaces determined by multiple constraints. The results yield asymptotically exact expressions that provide an accurate approximation of probabilities up to 1%, which considerably extends the applicability of large deviations results and enables efficient approximation of blocking probabilities for realistic mobile communications networks.

We investigate the sensitivity of the leaky-bucket filter, which is known as a traffic-shaping mechanism of source traffic entering an ATM network. Adopting the gradual input source model and using the infinitesimal perturbation analysis, we derive the strongly consistent estimators for the gradients of mean cell delay and a smoothness index of output process with respect to the design parameters of the system. Simulation results show the validity of our estimators.

In a renewal process, the interarrival times are independent and identically distributed, and in a renewal reward process, the pairs of reward and interarrival time are i.i.d. Many useful results hold for these processes. This paper relaxes the assumption of identical distributions while keeping the assumption of independence. This paper explores the properties of the mean average number of occurrence of events and the mean average reward on any finite time interval. The paper also discusses the limiting properties of these two quantities and extends many results from the renewal process and renewal reward process to the more general counting process and reward sequence.

We consider all-terminal reliability, one of the more popular models in the field of network reliability. A graph with n nodes and e edges, where the nodes are perfectly reliable and the edges survive independently with equal probability p, is said to be a uniformly optimally reliable graph if it has for all values of p (0 ≤ p ≤ 1) an equal or higher reliability among all graphs with the same number of nodes and edges. Boesch et al. [4] verified the existence of uniformly optimally reliable graphs for e = n − 1, e = n, e = n + 1, and e = n + 2; he has also given a conjecture for e = n + 3. Wang [8] proved this conjecture. In this article, we present four new infinite families of graphs that we conjecture to be uniformly optimal.

A simulation-based algorithm for learning good policies for a discrete-time stochastic control process with unknown transition law is analyzed when the state and action spaces are compact subsets of Euclidean spaces. This extends the Q-learning scheme of discrete state/action problems along the lines of Baker [4]. Almost sure convergence is proved under suitable conditions.

Motivated by many applications in production planning, system reliability, queueing networks, and wireless communication, this work is devoted to singularly perturbed Markov chains with finite states. Focusing on nonstationary processes with the inclusion of transient states, asymptotic error bounds of a sequence of suitably scaled occupation measures are derived. The main tools used include martingales and differential equations. The results are useful for analyzing structural properties of the underlying Markov chains and for designing nearly optimal and hierarchical controls of large-scale and complex systems.

To protect the Dutch polders against flooding, more than 2500 km of dikes have been constructed. Due to settlement, subsoil consolidation, and relative sea-level rise, these dikes slowly sink “away into the sea” and should therefore be heightened regularly (at present, every 50 years). In this respect, one is interested in safe and cost-optimal dike heightenings for which the sum of the initial cost of investment and the future (discounted) cost of maintenance is minimal.

For optimization purposes, a maintenance model has been developed for dikes subject to uncertain crest-level decline. On the basis of engineering knowledge, crest-level decline has been modeled as a monotone stochastic process with expected decline being either linear or nonlinear (i.e., linear after transformation) in time. For both models and for a particular unit time, the increments are distributed according to mixtures of exponentials.

In a case study, the maintenance decision model has been applied to the problem of heightening the Dutch “Oostmolendijk.”

In this note, we correct an error in the paper by Fu and Hu [1] for the perturbation analysis estimator given for the gradient of an American call option payoff on an underlying asset paying multiple dividends. We then introduce a different asset price model that is more straightforward than the previous model, and derive the corresponding gradient estimators. We conclude with a brief discussion of extensions of the estimator to other American-style options.

Numerical inversion of a one-sided z transform, corresponding to causal positive sequence, is considered. The numerical inversion requires the availability of a finite number of the transform's derivatives. The approximate analytical form is obtained by resorting to the maximum entropy principle. Entropy and then L1-norm convergence are proved.

We study the multichain case of a vector-valued Markov decision process with average reward criterion. We characterize optimal deterministic stationary policies via systems of linear inequalities and discuss a policy iteration algorithm for finding all optimal deterministic stationary policies.

In the proof of Theorem 2.6 in [1], we reduced the comparison of Ii(n) and Ii+1(n) to Ii−k(n − k) and Ii+1(n − k). Then, we mistakenly stated: “Suppose i − k > k, then we repeat the same reduction.” In fact, the reduction can be repeated only if i′ = n − k − i = i − k + 1. Consequently, Theorem 2.6 was proved only for t = 2. For similar reasons, Corollary 2.7 was proved only for t = 2, and Theorem 2.8 only for t = 1.