5 - Two-Player Zero-Sum Discounted Games

Summary

In this chapter, we extend the notion of discounted payoff to the model of stochastic games, and we define the concept of discounted equilibrium. We then prove that every two-player zero-sum stochastic game admits a discounted value, and that each player has a stationary discounted optimal strategy. The proof uses the same tools we employed in Chapter~\ref{section:mdp} to prove that in Markov decision problems the decision maker has a stationary discounted optimal strategy.

We finally prove that the discounted value is continuous in the parameters of the game, namely the payoff function, the transition function, and the discount factor.