In this paper, we consider denumerable-state continuous-time Markov decision processes with (possibly unbounded) transition and reward rates and general action space under the discounted criterion. We provide a set of conditions weaker than those previously known and then prove the existence of optimal stationary policies within the class of all possibly randomized Markov policies. Moreover, the results in this paper are illustrated by considering the birth-and-death processes with controlled immigration in which the conditions in this paper are satisfied, whereas the earlier conditions fail to hold.

A large deviation expansion is used to evaluate the impact of errors in a geographical database on the computation of travel times. We work in the framework of discrete random variables and improve a theorem by Book to solve this problem. Simulations are provided to illustrate the methodology.

The following problem in risk theory is considered. An insurance company, endowed with an initial capital a ≥ 0, receives premiums and pays out claims that occur according to a renewal process {N(t), t ≥ 0}. The times between consecutive claims are i.i.d. The sequence of successive claims is a sequence of i.i.d. random variables. The capital of the company is invested at interest rate α ∊ [0,1], claims increase at rate β ∊ [0,1]. The aim is to find the stopping time that maximizes the capital of the company. A dynamic programming method is used to find the optimal stopping time and to specify the expected capital at that time.

In a sequence of Markov-dependent trials, the optimal strategy which maximizes the probability of stopping on the last success is considered. Both homogeneous Markov chains and nonhomogeneous Markov chains are studied. For the homogeneous case, the analysis is divided into two parts and both parts are realized completely. For the nonhomogeneous case, we prove a result which contains the result of Bruss (2000) under an independence structure.

Asymptotic behaviour of the distribution of the absolute maximum of a process with independent increments is studied depending on the properties of the Lévy measure of the process. Some applications to the risk process are also considered.

Suppose that there is a sequence of programs or jobs that are scheduled to be executed one after another on a computer. A program may terminate its execution because of the failure of the computer, which will obliterate all work the computer has accomplished, and the program has to be run all over again. Hence, it is common to save the work just completed after the computer has been working for a certain amount of time, say y units. It is assumed that it takes a certain time to perform a save. During the saving process, the computer is still subject to random failure. No matter when the computer failure occurs, it is assumed that the computer will be repaired completely and the repair time will be negligible. If saving is successful, then the computer will continue working from the end of the last saved work; if the computer fails during the saving process, then only unsaved work needs to be repeated. This paper discusses the optimal work size y under which the long-run average amount of work saved is maximized. In particular, the case of an exponential failure time distribution is studied in detail. The properties of the optimal age-replacement policy are also derived when the work size y is fixed.

In this paper, we study ruin probabilities in two generalized risk models. The effects of timing of payments and interest on the ruin probabilities in the models are considered. The rates of interest are assumed to have a dependent autoregressive structure. Generalized Lundberg inequalities for the ruin probabilities are derived by a renewal recursive technique. An illustrative application is given to the compound binomial risk process.

An explicit convolution representation for the equilibrium residual lifetime distribution of compound zero-modified geometric distributions is derived. Second-order reliability properties are seen to be essentially preserved under geometric compounding, and complement results of Brown (1990) and Cai and Kalashnikov (2000). The convolution representation is then extended to the nth-order equilibrium distribution. This higher-order convolution representation is used to evaluate the stop-loss premium and higher stop-loss moments of the compound zero-modified geometric distribution, as well as the Laplace transform of the kth moment of the time of ruin in the classical risk model.

We consider an infinite capacity buffer where the incoming fluid traffic belongs to K different types modulated by K independent Markovian on-off processes. The kth input process is described by three parameters: (λk, μk, rk), where 1/λk is the mean off time, 1/μk is the mean on time, and rk is the constant peak rate during the on time. The buffer empties the fluid at rate c according to a first come first served (FCFS) discipline. The output process of type k fluid is neither Markovian, nor on-off. We approximate it by an on-off process by defining the process to be off if no fluid of type k is leaving the buffer, and on otherwise. We compute the mean on time τkon and mean off time τkoff. We approximate the peak output rate by a constant rko so as to conserve the fluid. We approximate the output process by the three parameters (λko, μko, rko), where λko = 1/τkoff, and μko = 1/τkon. In this paper we derive methods of computing τkon, τkoff and rko for k = 1, 2,…, K. They are based on the results for the computation of expected reward in a fluid queueing system during a first passage time. We illustrate the methodology by a numerical example. In the last section we conduct a similar output analysis for a standard M/G/1 queue with K types of customers arriving according to independent Poisson processes and requiring independent generally distributed service times, and following a FCFS service discipline. For this case we are able to get analytical results.

We consider an infinite-capacity buffer receiving fluid at a rate depending on the state of an M/M/1 queue. We obtain a new analytic expression for the joint stationary distribution of the buffer level and the state of the M/M/1 queue. This expression is obtained by the use of generating functions which are explicitly inverted. The case of a finite capacity fluid queue is also considered.

This paper is concerned with the correlation structure of a stationary discrete time-series with long memory or long-range dependence. Given a sequence of bounded variation, we obtain necessary and sufficient conditions for a function generated from the sequence to be a proper correlation function. These conditions are applied to derive various slowly decaying correlation models. To obtain correlation models with short-range dependence from an absolutely summable sequence, a simple method is introduced.

Uniform sequential tree-building aggregation of n particles is analyzed together with the effect of the avalanche that takes place when a subtree rooted at a uniformly chosen vertex is removed. For large n, the expected subtree size is found to be ≃ logn both for the tree of size n and the tree that remains after an avalanche. Repeated breakage-restoration cycles are seen to give independent avalanches which attain size k(1 ≤ k ≤ n-1) with probability (k(k+1))-1 and restored trees that are recursive.

There are two types of Asian options in the financial markets which differ according to the role of the average price. We give a symmetry result between the floating- and fixed-strike Asian options. The proof involves a change of numéraire and time reversal of Brownian motion. Symmetries are very useful in option valuation, and in this case the result allows the use of more established fixed-strike pricing methods to price floating-strike Asian options.

We introduce Galton-Watson-style branching processes in random environments which are deteriorating rather than stationary or independent. Some primary results on process growth and extinction probability are shown, and two simple examples are given.

For the M/G/c loss system, it is well known that Erlang's loss probability is convex in the number of servers. We extend this result firstly to renewal arrivals and exponential service, then to regenerative arrivals and exponential service, and finally to an arbitrary arrival process with i.i.d. service times that are independent of the arrival process.

We present a recursive method of computation for the probability that at most k customers were served during the busy period of an M/G/1 retrial queue.

We consider a network of dams to which the external input is a multivariate Markov additive process. For each state of the Markov chain modulating the Markov additive process, the release rates are linear (constant multiple of the content level). Each unit of material processed by a given station is then divided into fixed proportions each of which is routed to another station or leaves the system. For each state of the modulating process, this routeing is determined by some substochastic matrix. We identify simple conditions for stability and show how to compute transient and stationary characteristics of such networks.

Ebrahimi (2001) proposes an interesting model for assessing a system's reliability, expressing the failure time of the system in terms of a deterioration process and covariates. He also provides illustrative examples and gives some properties of the model. In this note, we give conditions for negative ageing of the system's lifetime, and we correct one of his statements. Moreover, for the lifetimes of two systems of the same kind, some stochastic comparisons are presented.

In this paper, we investigate k-out-of-n systems with independent and identically distributed components. Some characterizations of the IFR(2), DMRL, NBU(2) and NBUC classes of life distributions are obtained in terms of the monotonicity of the residual life given that the (n-k)th failure has occurred at time t ≥ 0. These results complement those reported by Belzunce, Franco and Ruiz (1999). Similar conclusions based on the residual life of a parallel system conditioned by the (n-k)th failure time are presented as well.

As proposed by Ebrahimi, uncertainty in the residual lifetime distribution can be measured by means of the Shannon entropy. In this paper, we analyse a dual characterization of life distributions that is based on entropy applied to the past lifetime. Various aspects of this measure of uncertainty are considered, including its connection with the residual entropy, the relation between its increasing nature and the DRFR property, and the effect of monotonic transformations on it.