A Markov decision chain with denumerable state space incurs two types of costs — for example, an operating cost and a holding cost. The objective is to minimize the expected average operating cost, subject to a constraint on the expected average holding cost. We prove the existence of an optimal constrained randomized stationary policy, for which the two stationary policies differ on at most one state. The examples treated are a packet communication system with reject option and a single-server queue with service rate control.

In this article, we give several results on (multivariate and univariate) stochastic comparisons of generalized order statistics. We give conditions on the underlying distributions and the parameters on which the generalized order statistics are based, to obtain stochastic comparisons in the stochastic, dispersive, hazard rate, and likelihood ratio orders. Our results generalize some recent results for order statistics, record values, and generalized order statistics and provide some new results for other models such as k-record values and order statistics under multivariate imperfect repair.

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that these probabilities are known or believed to belong to and by allowing the probabilities to vary over such sets. This leads to the definition of an imprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-called lower and upper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at time n evolves as n→∞: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalization of the classical Perron–Frobenius theorem to imprecise Markov chains.

Motivated by the patient triage problem in emergency response, we consider a single-server clearing system in which jobs might abandon the system if they are not taken into service within their “lifetime.” In this system, jobs are characterized by their lifetime and service time distributions. Our objective is to dynamically determine the optimal or near-optimal order of service for jobs so as to minimize the total number of abandonments. We first show that if the jobs can be ordered in such a way that the job with the shortest lifetime (in the sense of hazard rate ordering) also has the shortest service time (in the sense of likelihood ratio ordering), then the optimal policy gives the highest priority to this “time-critical” job independently of the system state. For the case in which the jobs with shorter lifetimes have longer service times, we observed that the optimal policy generally has a complex structure that might depend on the type and number of jobs available. For this case, we provide partial characterizations of the optimal policy and obtain sufficient conditions under which a state-independent policy is optimal. Furthermore, we develop two state-dependent heuristic policies, and by means of a numerical study, we show that these heuristics perform well, especially when jobs abandon the system at a relatively faster rate when compared to service rates. Based on our analytical and numerical results, we develop several insights on patient triage in the immediate aftermath of a mass casualty event. For example, we conclude that in a worst-case scenario, where medical resources are overwhelmed with a large number of casualties who need immediate attention, it is crucial to implement state-dependent policies such as the heuristic policies proposed in this article.

We obtain some techniques to study the shape of reliability functions (failure rate, mean residual life, etc.) by using the s-equilibrium distribution of a renewal process defined by Fagiuoli and Pellerey (Naval Res. Logist., 1993). We apply these techniques to study how to obtain distributions with bathtub shaped failure rate (BFR) from mixtures of two positive truncated normal distributions.

The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P(·) and ergodic matrix Π is the matrix D ≡ ∫0∞(P(t) − Π) dt. We give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth–death process is investigated in detail. We also describe a new application of deviation matrices by showing that a measure for the convergence to stationarity of a stochastically increasing Markov chain can be expressed in terms of the elements of the deviation matrix of the chain.

We consider a single-item, periodic-review inventory model with uncertain demands in which each period's production volume is limited by a capacity level. The demand distributions, capacity levels, and cost parameters vary according to a periodic pattern. We prove that modified base-stock policies are optimal for the finite-horizon planning model and for both the infinite-horizon discounted and undiscounted cost criterion. We further show that the optimal base-stock levels can be calculated via a simple but efficient value-iteration method. Finally, we have conducted a comprehensive numerical study to ascertain the efficiency of this solution method as well as various qualitative properties of the performance of capacitated production/inventory systems under periodically varying demand and cost patterns.

A random intersection graph is constructed by assigning independently to each vertex a subset of a given set and drawing an edge between two vertices if and only if their respective subsets intersect. In this article a model is developed in which each vertex is given a random weight and vertices with larger weights are more likely to be assigned large subsets. The distribution of the degree of a given vertex is characterized and is shown to depend on the weight of the vertex. In particular, if the weight distribution is a power law, the degree distribution will be as well. Furthermore, an asymptotic expression for the clustering in the graph is derived. By tuning the parameters of the model, it is possible to generate a graph with arbitrary clustering, expected degree, and—in the power-law case—tail exponent.

The burn-in problem is reframed in a decision-making context. The role of the predictive life distribution in the burn-in decision problem is highlighted. The case in which the predictive distribution is a mixture of exponentials is discussed extensively.

In the present article we provide a formula that facilitates the evaluation of the signature of a reliability structure by a generating function approach. A simple sufficient condition is also derived for proving the nonpreservation of the IFR property for the system's lifetime (when the components are IFR) by exploiting the signature of the system. As an application of the general results, we deduce recurrence relations for the signature of a linear consecutive k-out-of-n: F system. We establish a simple relation between the signature of a linear and a circular system and investigate the IFR preservation property under the formulation of such systems.

We define a notion of regularity ordering among stochastic processes called directionally convex (dcx) ordering and give examples of doubly stochastic Poisson and Markov renewal processes where such ordering is prevalent. Further-more, we show that the class of segmented processes introduced by Chang, Chao, and Pinedo [3] provides a rich set of stochastic processes where the dcx ordering can be commonly encountered. When the input processes to a large class of queueing systems (single stage as well as networks) are dcx ordered, so are the processes associated with these queueing systems. For example, if the input processes to two tandem /M/c1→/M/c2→…→/M/cm queueing systems are dcx ordered, so are the numbers of customers in the systems. The concept of directionally convex functions (Shaked and Shanthikumar [15]) and the notion of multivariate stochastic convexity (Chang, Chao, Pinedo, and Shanthikumar [4]) are employed in our analysis.

This article focuses on simulating fractional Brownian motion (fBm). Despite the availability of several exact simulation methods, attention has been paid to approximate simulation (i.e., the output is approximately fBm), particularly because of possible time savings. In this article, we study the class of approximate methods that are based on the spectral properties of fBm's stationary incremental process, usually called fractional Gaussian noise (fGn). The main contribution is a proof of asymptotical exactness (in a sense that is made precise) of these spectral methods. Moreover, we establish the connection between the spectral simulation approach and a widely used method, originally proposed by Paxson, that lacked a formal mathematical justification. The insights enable us to evaluate the Paxson method in more detail. It is also shown that spectral simulation is related to the fastest known exact method.

The purpose of this article is to study several preservation properties of the mean inactivity time order under the reliability operations of convolution, mixture, and shock models. In that context, the increasing mean inactivity time class of lifetime distributions is characterized by means of right spread order and increasing convex order. Some applications in reliability theory are described. Finally, a new test of such a class is discussed.

Inactivity time is a reliability conception dual to the residual life. In this article, we establish some stochastic comparisons on inactivity time and the residual life of series and parallel systems, respectively. Some applications are presented as well.

This paper considers a single machine and n jobs. The machine is subject to breakdown and repair. Job j has a weight wj, a random processing time Xj, and is available for processing from a random time Rj on. The jobs may be subject to precedence constraints. We are interested in optimal policies that minimize the following objective functions: (i) the weighted sum of the completion times; (ii) the weighted sum of an exponential function of the completion times; and (iii) the weighted number of late jobs having due dates.

Recently, an alternative measure of uncertainty called extropy is proposed by Lad et al. [12]. The extropy is a dual of entropy which has been considered by researchers. In this article, we introduce an alternative measure of uncertainty of random variable which we call it cumulative residual extropy. This measure is based on the cumulative distribution function F. Some properties of the proposed measure, such as its estimation and applications, are studied. Finally, some numerical examples for illustrating the theory are included.

Semi-Markov decision processes underlie the control of many queueing systems. In this paper, we deal with infinite state semi-Markov decision processes with nonnegative, unbounded costs and finite action sets. Axioms for the existence of an expected average cost optimal stationary policy are presented. These conditions generalize the work in Sennott [22] for Markov decision processes. Verifiable conditions for the axioms to hold are obtained. The theory is applied to control of the M/G/l queue with variable service parameter, with on-off server, and with batch processing, and to control of the G/M/m queue with variable arrival parameter and customer rejection. It is applied to a timesharing network of queues with a single server and finally to optimal routing of Poisson arrivals to parallel exponential servers. The final section extends the existence result to compact action spaces.

A set of ni arms of type i, i = 1,…, L, is available. A pull of arm of type i occupies a duration Vi at the end of which a reward Ci and Ni1,…, NiL new arms are obtained, while all other arms are frozen. A Gittins priority order of types is obtained and shown to yield the maximal discounted reward from this branching process of arms.

We consider the optimal control of two parallel servers in a two-stage tandem queuing system with two flexible servers. New jobs arrive at station 1, after which a series of two operations must be performed before they leave the system. Holding costs are incurred at rate h1 per unit time for each job at station 1 and at rate h2 per unit time for each job at station 2.

The system is considered under two scenarios; the collaborative case and the noncollaborative case. In the prior, the servers can collaborate to work on the same job, whereas in the latter, each server can work on a unique job although they can work on separate jobs at the same station. We provide simple conditions under which it is optimal to allocate both servers to station 1 or 2 in the collaborative case. In the noncollaborative case, we show that the same condition as in the collaborative case guarantees the existence of an optimal policy that is exhaustive at station 1. However, the condition for exhaustive service at station 2 to be optimal does not carry over. This case is examined via a numerical study.

This paper considers routing to parallel queues in which each queue has its own single server and service times are exponential with nonidentical parameters. We give conditions on the cost function such that the optimal policy assigns customers to a faster queue when that server has a shorter queue. The queues may have finite buffers, and the arrival process can be controlled and can depend on the state and routing policy. Hence our results on the structure of the optimal policy are also true when the assigning control is in the “last” node of a network of service centers. Using dynamic programming we show that our optimality results are true in distribution.