We prove the conjecture formulated in Litvak and Ejov (2009), namely, that the trace of the fundamental matrix of a singularly perturbed Markov chain that corresponds to a stochastic policy feasible for a given graph is minimised at policies corresponding to Hamiltonian cycles.

An open hierarchical (manpower) system divided into a totally ordered set of k grades is discussed. The transitions occur only from one grade to the next or to an additional (k+1)th grade representing the external environment of the system. The model used to describe the dynamics of the system is a continuous-time homogeneous Markov chain with k+1 states and infinitesimal generator R = (rij) satisfying rij = 0 if i > j or i + 1 < j ≤ k (i, j = 1,…,k+1), the transition matrix P between times 0 and 1 being P = expR. In this paper, two-wave panel data about the hierarchical system are considered and the resulting fact that, in general, the maximum-likelihood estimated transition matrix cannot be written as an exponential of an infinitesimal generator R having the form described above. The purpose of this paper is to investigate when this can be ascribed to the effect of sampling variability.

In this paper we extend the class of zero-order threshold autoregressive models to a much richer class of mixture models. The new class has the important property of duality which, as we show, corresponds to time reversal. We are then able to obtain the time reversals of the zero-order threshold models and to characterise the time-reversible members of this subclass. These turn out to be quite trivial. The time-reversible models of the more general class do not suffer in this way. The complete stationary distributional structure is given, as are various moments, in particular the autocovariance function. This is shown to be of ARMA type. Finally we give two examples, the second of which extends from the finite to the countable mixture case. The general theory for this extension will be given elsewhere.

Let A be a stochastic matrix and ε a positive number. We consider all stochastic matrices within ε of A and their corresponding stochastic eigenvectors. A convex polytope containing these vectors is described. An efficient algorithm for computing bounds on the components of these vectors is also given. The work is compared to previous such work done by the author and by Courtois and Semai.

Paz (1963), (1971) has shown that for an n × n stochastic matrix P the property that some power of P has a positive ergodic coefficient is decidable. In particular he shows that α (Pk) > 0 for some k, only if it is positive for k = ½n(n – 1). However he states that it is not known whether this bound is sharp. In this paper a sharp bound is given, namely k =½(n – 1)2 + ½ or ½(n − 1)2 + 1 depending on whether n is even or odd. The proof of this is based on some results from number theory.

If an n × n stochastic matrix has a column with no zeros, one can immediately conclude that the chain is ergodic and the state corresponding to that column is persistent and aperiodic. In this paper it is shown that it is decidable whether or not some power of a finite stochastic matrix has a positive column. Some problems regarding positive columns in infinite stochastic matrices are also considered.

A discrete time Markov chain with (n + 1) states is constructed in which there are n states having different decay parameters for geometric ergodicity.