It is shown that, in a superposition of finitely many independent renewal processes, an observation from the limiting (when t →∞) joint distribution of backward and forward recurrence times at t can be simulated by simulating an observation of the pair (UW, (1 – U)W), where U and Ware independent random variables with U ~ uniform(0, 1) and W distributed according to the limiting total life distribution of the superposition process.

This paper is concerned with a discrete-time population model in which a new individual entering the population at time t can produce offspring for the first time at time t + 2 and then subsequently at times t + 3, t + 4, ···. The numbers of offspring produced on each occasion are independent random variables each with the distribution of Z for which EZ = m <∞, and individuals have independent lines of descent. This model is contrasted with the corresponding Bienaymé-Galton-Watson one. If Xn denotes the number of individuals in the population at time n, it is shown that z–nXn almost surely converges to a random variable W, as n→∞, where Various properties of W are obtained, in particular W > 0 a.s. if and only if EZ | log Z | < ∞ Results are also given on the rate of convergence of to z when Var Z < ∞ and these display a surprising dependence on the size of z.

We consider a system that gradually deteriorates. At random times Tn the system is inspected and may or may not be replaced, but it must be if its state is too bad. The process of deterioration is described by a Markovian renewal process. Under some monotonicity conditions we derive a policy for the finite and the infinite horizon which minimizes the total discounted costs. This paper is a generalization of a result of Feldman (1977).

Existing results on the asymptotic behaviour of solutions to two dynamic programming equations involving non-negative matrices are reviewed and strengthened in certain directions. The results are then applied to strategies for harvesting of a small population so as to optimise its survival potential in a limited environment.

Recently Gani (1965) has considered a stochastic model for the attachment of phages to bacteria. In this paper we study some of the basic properties of this model. These results will be used in a subsequent paper dealing with the estimation of the parameter which occurs in this model.

In this paper, we study the new better than used in expectation (NBUE) and new worse than used in expectation (NWUE) properties of Markov renewal processes. We show that a Markov renewal process belongs to a more general class of stochastic processes encountered in reliability or maintenance applications. We present sufficient conditions such that the first-passage times of these processes are new better than used in expectation. The results are applied to the study of shock and repair models, random repair time processes, inventory, and queueing models.

This study is concerned with finite Markov decision processes whose dynamics and reward structure are unknown but the state is observable exactly.

We establish a learning algorithm which yields an optimal policy and construct an adaptive policy which is optimal under the average expected reward criterion.

Optimal stopping rules under various conditions are obtained for a proofreader who has a probability p (known or unknown) of detecting a misprint in proofsheets which contain an unknown but Poisson-distributed number of misprints.

For a sequence ξ1, ξ2, · ·· of i.i.d. random variables let X0 = 0 and Xk = max(Xk–1 + ξ k, 0) for k = 1, 2, ···. Let . These stopping times are used in Page's (1954) one-sided cusum procedures and are also important in queueing theory. Various asymptotic properties of Nx are derived.

An urn contains black and red balls. Let Z n be the proportion of black balls at time n and 0≤L<U≤1 random barriers. At each time n, a ball b n is drawn. If b n is black and Z n-1<U, then b n is replaced together with a random number B n of black balls. If b n is red and Z n-1>L, then b n is replaced together with a random number R n of red balls. Otherwise, no additional balls are added, and b n alone is replaced. In this paper we assume that R n =B n . Then, under mild conditions, it is shown that Z n →a.s. Z for some random variable Z, and D n ≔√n(Z n -Z)→𝒩(0,σ2) conditionally almost surely (a.s.), where σ2 is a certain random variance. Almost sure conditional convergence means that ℙ(D n ∈⋅|𝒢n )→w 𝒩(0,σ2) a.s., where ℙ(D n ∈⋅|𝒢n ) is a regular version of the conditional distribution of D n given the past 𝒢n . Thus, in particular, one obtains D n →𝒩(0,σ2) stably. It is also shown that L<Z<U a.s. and Z has nonatomic distribution.

Nanosystems are devices that are in the size range of a billionth of a meter (1 x 10-9) and therefore are built necessarily from individual atoms. The one-dimensional nanosystems or linear nanosystems cover all the nanosized systems which possess one dimension that exceeds the other two dimensions, i.e. extension over one dimension is predominant over the other two dimensions. Here only two of the dimensions have to be on the nanoscale (less than 100 nanometers). In this paper we consider the structural relationship between a linear nanosystem and its atoms acting as components of the nanosystem. Using such information, we then assess the nanosystem's limiting reliability which is, of course, probabilistic in nature. We consider the linear nanosystem at a fixed moment of time, say the present moment, and we assume that the present state of the linear nanosystem depends only on the present states of its atoms.

Correction terms for the diffusion approximation to the maximum and ruin probabilities for a random walk with small negative drift, obtained by Siegmund (1979) in the exponential family case, are extended by different methods to some non-exponential family cases.

(1.1) In this paper, acquaintance will be assumed with the basic mechanics of phage/bacterium, antibody/virus interactions; for background reading see, for example, Adams (1959), Bouanchaud (1970), Durham and King (1969) and Fraser (1967).

In this paper, we discuss the problem of the pricing of American-style options in the exponential Lévy security market model. This model is typically incomplete, and we derive the explicit bounds of the interval of no arbitrage prices and the related optimal stopping moments for American put options and American call options in both finite and infinite horizon time. We consider a large class of Lévy processes.

This paper considers the best-choice problem with a random number of objects having a known distribution. The optimality equation of the problem reduces to an integral equation by a scaling limit. The equation is explicitly solved under conditions on the distribution, which relate to the condition for an OLA policy to be optimal in Markov decision processes. This technique is then applied to three different versions of the problem and an exact value for the asymptotic optimal strategy is found.

We study a family of Markov processes on P (k), the space of partitions of the natural numbers with at most k blocks. The process can be constructed from a Poisson point process on R + x ∏i=1 k P (k) with intensity dt ⊗ ϱν (k), where ϱν is the distribution of the paintbox based on the probability measure ν on P m, the set of ranked-mass partitions of 1, and ϱν (k) is the product measure on ∏i=1 k P (k). We show that these processes possess a unique stationary measure, and we discuss a particular set of reversible processes for which transition probabilities can be written down explicitly.

Motivated by a problem in neural encoding, we introduce an adaptive (or real-time) parameter estimation algorithm driven by a counting process. Despite the long history of adaptive algorithms, this kind of algorithm is relatively new. We develop a finite-time averaging analysis which is nonstandard partly because of the point process setting and partly because we have sought to avoid requiring mixing conditions. This is significant since mixing conditions often place restrictive history-dependent requirements on algorithm convergence.

Here we introduce two simple models: simple cubic random packing and random packing by Hamming distance. Consider the packing density γ d of dimension d by cubic random packing. From computer simulations up to dimension 11, γ d+1/γ d seems to approach 1. Also, we give simulation results for random packing by Hamming distance and discuss the behavior of packing density when dimensionality is increased. For the case of Hamming distances of 2 or 3, d–α fits the simulation results of packing density where α is an empirical constant. The variance of packing density is larger when k is even and smaller when k is odd, where k represents Hamming distance.

The study of simple solar energy storage models leads to the question of analyzing the equilibrium distribution of Markov chains (Harris chains), for which the state at epoch (n + 1) (i.e. the temperature of the storage tank) depends on the state at epoch n and on a controlled input, acceptance of which entails a further decrease of the temperature level. Here we study the model where the input is exponentially distributed. For all values of the parameters involved an explicit expression for the equilibrium distribution of the Markov chain is derived, and from this we calculate, as one of the possible applications, the exact values of the mean of this equilibrium distribution.

The problem of determining the asymptotic form of the stock vector n(t) in a continuous time Markovian manpower model is solved for asymptotically exponential recruitment functions {R(t)}. A new approach to the limiting behaviour of some manpower systems with given total sizes {N(t)} is then given by means of time-inhomogeneous Markov processes.