We consider a two-dimensional reflecting random walk on the non-negative integer quadrant. It is assumed that this reflecting random walk has skip-free transitions. We are concerned with its time-reversed process assuming that the stationary distribution exists. In general, the time-reversed process may not be a reflecting random walk. In this paper, we derive necessary and sufficient conditions for the time-reversed process also to be a reflecting random walk. These conditions are different from but closely related to the product form of the stationary distribution.

In this paper, we provide a new technique for analyzing the nonstationary Erlang loss queueing model with abandonment. Our method uniquely combines the use of the functional Kolmogorov forward equations with the well-known Gram-Charlier series expansion from the statistics literature. Using the Gram-Charlier series expansion, we show that we can estimate salient performance measures of the loss queue such as the mean, variance, skewness, kurtosis, and blocking probability. Lastly, we provide numerical examples to illustrate the effectiveness of our approximations.

Generally, the multi-armed has been studied under the setting that at each time step over an infinite horizon a controller chooses to activate a single process or bandit out of a finite collection of independent processes (statistical experiments, populations, etc.) for a single period, receiving a reward that is a function of the activated process, and in doing so advancing the chosen process. Classically, rewards are discounted by a constant factor β∈(0, 1) per round.

In this paper, we present a solution to the problem, with potentially non-Markovian, uncountable state space reward processes, under a framework in which, first, the discount factors may be non-uniform and vary over time, and second, the periods of activation of each bandit may be not be fixed or uniform, subject instead to a possibly stochastic duration of activation before a change to a different bandit is allowed. The solution is based on generalized restart-in-state indices, and it utilizes a view of the problem not as “decisions over state space” but rather “decisions over time”.

We analyze optimal replacement and repair problems of semi-Markov missions that are composed of phases with random sequence and durations. The mission process is the minimal semi-Markov process associated with a Markov renewal process. The system is a complex one consisting of non-identical components whose failure properties depend on the mission process. We prove some monotonicity properties for the optimal replacement policy and analyze the optimal repair problem under different cost structures.

Recently, Chen, Hsiau & Yang [1] proposed a new two-urn model with red and white balls and showed that the fractions of red balls in both urns converge almost surely to the same limit. We extend the results for the two-urn model to the q-urn model (q≥3) with similar dynamics of drawing and adding balls. We use matrix forms and martingale theory to show that the fractions of red balls in all urns converge almost surely to the same limit.

In this paper, we study the instant availability A(t) of a repairable system by integral equation. We have proved initial monotonicity of availability, and derived lower bounds of A(t) and average availability. The availabilities of two systems are compared.

We study mathematically a method for discovering which gene is related to a cell characteristic (“phenotype”) of interest. The method is based on RNA interference – a molecular process for gene deactivation – and on coupling the phenotype with cell fluorescence. A small number of candidate genes are thus isolated, and then tested individually. We model probabilistically this process, prove a limit theorem for its outcome, and derive operational guidelines for maximizing the probability of successful gene discovery.