We consider the stochastic sequence {Yt}t∈ℕ defined recursively by the linear relation Yt+1=AtYt+Bt in a random environment. The environment is described by the stochastic process {(At,Bt)}t∈ℕ and is under the simultaneous control of several agents playing a discounted stochastic game. We formulate sufficient conditions on the game which ensure the existence of Nash equilibria in Markov strategies which have the additional property that, in equilibrium, the process {Yt}t∈ℕ converges in distribution to a stationary regime.

We study the long-run behaviour of interactive Markov chains on infinite product spaces. The behaviour at a single site is influenced by the local situation in some neighbourhood and by a random signal about the average situation throughout the whole system. The asymptotic behaviour of such Markov chains is analyzed on the microscopic level and on the macroscopic level of empirical fields. We give sufficient conditions for convergence on the macroscopic level. Combining a convergence result from the theory of random systems with complete connections with a perturbation of the Dobrushin-Vasserstein contraction technique, we show that macroscopic convergence implies that the underlying microscopic process has local asymptotic loss of memory.

In this paper we study the so-called random coeffiecient autoregressive models (RCA models) and (generalized) autoregressive models with conditional heteroscedasticity (ARCH/GARCH models). Both models can be represented as random systems with complete connections. Within this framework we are led (under certain conditions) to CL-regular Markov processes and we will give conditions under which (i) asymptotic stationarity, (ii) a law of large numbers and (iii) a central limit theorem can be shown for the corresponding models.

The classical exponential smoothing procedure to be applied to a sequence of observations is modified into some adaptive variants and also generalized to the recursive scheme

This means that given the history of observations and smoothed values up to time n, the updated smoothed value Wn +1 is given by a function u of Wn and Xn, which satisfies only certain structural properties. We study these procedures within the framework of random systems with complete connections and exploit results from Markov process theory to study the sequence . A broad class of ‘smoothing functions' u, which induce sequences with nice statistical properties, is presented. A method to estimate the limit of the expected smoothed values is also developed.,

One general and three specialized models of the Bush–Mosteller type are presented to describe the kind of learning experiment where the response of the learner is always reinforced. Inhomogeneity is admitted. The random sequences of response probabilities and of responses associated with the different models are considered. Information about the existence and the distribution of asymptotic response probabilities is provided. The stress is on sufficient and necessary conditions for convergence (a.s. or with positive probability) of the response sequence, which is what ‘learning' means.