This paper establishes new Foster-type criteria for a Markov chain on a general state space to be Harris recurrent, positive Harris recurrent, or geometrically ergodic. The criteria are based on drift conditions involving stopping times rather than deterministic steps. Meyn and Tweedie (1994) developed similar criteria involving random-sized steps, independent of the Markov chain under study. They also posed an open problem of finding criteria involving stopping times. Our results essentially solve that problem. We also show that the assumption of ψ-irreducibility is not needed when stating our drift conditions for positive Harris recurrence or geometric ergodicity.

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator.

Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula.

We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case.

We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

We study two FIFO single-server queueing models in which both the arrival and service processes are modulated by the amount of work in the system. In the first model, the nth customer's service time, Sn, depends upon their delay, Dn, in a general Markovian way and the arrival process is a non-stationary Poisson process (NSPP) modulated by work, that is, with an intensity that is a general deterministic function g of work in system V(t). Some examples are provided. In our second model, the arrivals once again form a work-modulated NSPP, but, each customer brings a job consisting of an amount of work to be processed that is i.i.d. and the service rate is a general deterministic function r of work. This model can be viewed as a storage (dam) model (Brockwell et al. (1982)), but, unlike previous related literature, (where the input is assumed work-independent and stationary), we allow a work-modulated NSPP. Our approach involves an elementary use of Foster's criterion (via Tweedie (1976)) and in addition to obtaining new results, we obtain new and simplified proofs of stability for some known models. Using further criteria of Tweedie, we establish sufficient conditions for the steady-state distribution of customer delay and sojourn time to have finite moments.