We consider residue expansions for survival and density/mass functions of first-passage distributions in finite-state semi-Markov processes (SMPs) in continuous and integer time. Conditions are given which guarantee that the residue expansions for these functions have a dominant exponential/geometric term. The key condition assumes that the relevant states for first passage contain an irreducible class, thus ensuring the same sort of dominant exponential/geometric terms as one gets for phase-type distributions in Markov processes. Essentially, the presence of an irreducible class along with some other conditions ensures that the boundary singularity b for the moment generating function (MGF) of the first-passage-time distribution is a simple pole. In the continuous-time setting we prove that b is a dominant pole, in that the MGF has no other pole on the vertical line $\{\text{Re}(s)=b\}.$ In integer time we prove that b is dominant if all holding-time mass functions for the SMP are aperiodic and non-degenerate. The expansions and pole characterisations address first passage to a single new state or a subset of new states, and first return to the starting state. Numerical examples demonstrate that the residue expansions are considerably more accurate than saddlepoint approximations and can provide a substitute for exact computation above the 75th percentile.

A finite-horizon insurance model is studied where the risk/reserve process can be controlled by reinsurance and investment in the financial market. Our setting is innovative in the sense that we describe in a unified way the timing of the events, that is, the arrivals of claims and the changes of the prices in the financial market, by means of a continuous-time semi-Markov process which appears to be more realistic than, say, classical diffusion-based models. Obtaining explicit optimal solutions for the minimizing ruin probability is a difficult task. Therefore we derive a specific methodology, based on recursive relations for the ruin probability, to obtain a reinsurance and investment policy that minimizes an exponential bound (Lundberg-type bound) on the ruin probability.

In this paper we consider the following problem. An angler buys a fishing ticket that allows him/her to fish for a fixed time. There are two locations to fish at the lake. The fish are caught according to a renewal process, which is different for each fishing location. The angler's success is defined as the difference between the utility function, which is dependent on the size of the fish caught, and the time-dependent cost function. These functions are different for each fishing location. The goal of the angler is to find two optimal stopping times that maximize his/her success: when to change fishing location and when to stop fishing. Dynamic programming methods are used to find these two optimal stopping times and to specify the expected success of the angler at these times.

Markov and semi-Markov processes with block tridiagonal transition matrices for their embedded discrete-time Markov chains are underlying stochastic models in many applied probability problems. In particular, identity-by-descent (IBD) problems for uncle-type and cousin-type relationships fall into this class. More specifically, the exact distributions of relevant IBD statistics for two individuals in either an uncle-type or cousin-type relationship are of interest. Such statistics are the amount of genome shared IBD by the two related individuals on a chromosomal segment and the number of IBD pieces on such a segment. These lead to special reward distributions associated with block tridiagonal transition matrices for continuous-time Markov chains. A method is provided for calculating explicit, closed-form expressions for Laplace transforms of general reward functions for such Markov chains. Some calculation results on the cumulative probabilities of relevant IBD statistics via a numerical inversion of the Laplace transforms are also provided for uncle/nephew and first-cousin relationships.

Aven and Jensen (1999) proposed an approximation to the downtime distribution for arbitrary coherent systems with binary and independent repairable components, where each component follows an alternating renewal process. The approximation is based on a mixture of the duration distributions of the cut sets in the system. We extend this approach to coherent systems composed by multistate independent components, where every component follows a semi-Markov process and where we introduce the concept of a minimal cut set in the multistate setting. We test our proposal on a simple power demand-generation system, by comparing the analytic approximation with simulation results, and we find it is accurate when the system is highly available.

In this paper we, firstly, present a recursive formula of theempirical estimator of the semi-Markov kernel. Then a non-parametricestimator of the expected cumulative operational time forsemi-Markov systems is proposed. The asymptotic properties of thisestimator, as the uniform strongly consistency and normality aregiven. As an illustration example, we give a numerical application.

The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distribution result for rewards accumulated over deterministic time intervals [0, t], even in the special case of continuous-time Markov chains. The present paper provides neat general results which lead to explicit closed-form expressions for the relevant Laplace transforms of general reward functions on semi-Markov and Markov additive processes.

We extend the definition of level-crossing ordering of stochastic processes, proposed by Irle and Gani (2001), to the case in which the times to exceed levels are compared using an arbitrary stochastic order, and work, in particular, with integral stochastic orders closed for convolution. Using a sample-path approach, we establish level-crossing ordering results for the case in which the slower of the processes involved in the comparison is skip-free to the right. These results are specially useful in simulating processes that are ordered in level crossing, and extend results of Irle and Gani (2001), Irle (2003), and Ferreira and Pacheco (2005) for skip-free-to-the-right discrete-time Markov chains, semi-Markov processes, and continuous-time Markov chains, respectively.

A simple, widely applicable method is described for determining factorial moments of N̂t, the number of occurrences in (0,t] of some event defined in terms of an underlying Markov renewal process, and asymptotic expressions for these moments as t → ∞. The factorial moment formulae combine to yield an expression for the probability generating function of N̂t, and thereby further properties of such counts. The method is developed by considering counting processes associated with events that are determined by the states at two successive renewals of a Markov renewal process, for which it both simplifies and generalises existing results. More explicit results are given in the case of an underlying continuous-time Markov chain. The method is used to provide novel, probabilistically illuminating solutions to some problems arising in the stochastic modelling of ion channels.

We derive a tight perturbation bound for hidden Markov models. Using this bound, we show that, in many cases, the distribution of a hidden Markov model is considerably more sensitive to perturbations in the emission probabilities than to perturbations in the transition probability matrix and the initial distribution of the underlying Markov chain. Our approach can also be used to assess the sensitivity of other stochastic models, such as mixture processes and semi-Markov processes.

In the reliability theory, the availability ofa component, characterized by non constant failure and repair rates,is obtained, at a given time, thanks to the computation of the marginal distributions of asemi-Markov process. These measures are shown to satisfy classicaltransport equations, the approximation of which can be donethanks to a finite volume method.Within a uniqueness result for the continuous solution,the convergence of the numerical scheme isthen proven in the weak measure sense,and some numerical applications, which show the efficiency and theaccuracy of the method, are given.

The aim of this paper is to define the entropy of a finite semi-Markov process. We define the entropy of the finite distributions of the process, and obtain explicitly its entropy rate by extending the Shannon–McMillan–Breiman theorem to this class of nonstationary continuous-time processes. The particular cases of pure jump Markov processes and renewal processes are considered. The relative entropy rate between two semi-Markov processes is also defined.

The formation of patterns from letters of a finite alphabet is considered. The strings of letters are generated by general discrete- and continuous-time models which embrace as particular cases all models considered in the literature. The letters of the alphabet are identified by the states of either discrete- or continuous-time semi-Markov processes. A new and unifying method is introduced for evaluation of the generating functions of both the intersite distance between occurrences of an arbitrary, but fixed, pattern and the waiting time until the first occurrence of that pattern. Our method also covers in a unified way relevant and important joint generating functions. Furthermore, our results lead to an easy and efficient implementation of the relevant evaluations.

We study linear jump parameter systems of differential and difference equations whose coefficients depend on the state of a semi-Markov process. We derive systems of equations for the first two moments of the random solutions of these jump parameter systems, and illustrate how moment equations can be used in examining their asymptotic stability.

The authors prove that certain reliability formulas which link asymptotic availability, mean normal operation time, mean time between failures, mean number of failures over a period of time and asymptotic Vesely rate, and which are well known in the case of modelling using a Markov jump process or an alternating renewal process, are also true in the context of more general modelling.

We suggest a new universal method of stochastic simulation, allowing us to generate rather efficiently random vectors with arbitrary densities in a connected open region or on its boundary. Our method belongs to the class of dynamic Monte Carlo procedures and is based on a special construction of a Markov chain on the boundary of the region. Its remarkable feature is that this chain admits a simple simulation, based on a universal (depending only on the dimensionality of the space) stochastic driver.

In a counting process considered at time t the focus is often on the length of the current interarrival time, whereas points in the past may be said to constitute information about the process. The paper introduces new concepts on how to quantify predictability of the future behavior of counting processes based on the past information and considers then situations in which the future points become more (or less) predictable. Various properties of our proposed concepts are studied and applications relevant to the reliability of repairable systems are given.

In this paper we introduce and define for the first time the concept of a non-homogeneous semi-Markov system (NHSMS). The problem of finding the expected population stucture is studied and a method is provided in order to find it in closed analytic form with the basic parameters of the system. Moreover, the problem of the expected duration structure in the state is studied. It is also proved that all maintainable expected duration structures by recruitment control belong to a convex set the vertices of which are specified. Finally an illustration is provided of the present results in a manpower system.

A wide class of multichannel queueing models appears to be useful in practice where the input stream of customers can be controlled at the moments preceding the customers' departures from the source (e.g. airports, transportation systems, inventories, tandem queues). In addition, the servicing facility can govern the intensity of the servicing process that further improves flexibility of the system. In such a multichannel queue with infinite waiting room the queueing process {Zt; t ≧ 0} is under investigation. The author obtains explicit formulas for the limiting distribution of (Zt) partly using an approach developed in previous work and based on the theory of semi-regenerative processes. Among other results the limiting distributions of the actual and virtual waiting time are derived. The input stream (which is not recurrent) is investigated, and distribution of the residual time from t to the next arrival is obtained. The author also treats a Markov chain embedded in (Zt) and gives a necessary and sufficient condition for its existence. Under this condition the invariant probability measure is derived.

An expression for the asymptotic or steady-state performance function is derived for a multistate coherent system when each component changes states in time according to a semi-Markov process, the stochastic processes being mutually independent. This generalizes the expression for system availability of a binary coherent system when the components are governed by mutually independent alternating renewal processes.