We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
We continue our discussion of hidden Markov models (HMMs) and consider in this chapter the solution of decoding problems. Specifically, given a sequence of observations , we would like to devise mechanisms that allow us to estimate the underlying sequence of state or latent variables
. That is, we would like to recover the state evolution that “most likely” explains the measurements. We already know how to perform decoding for the case of mixture models with independent observations by using (38.12a)–(38.12b). The solution is more challenging for HMMs because of the dependency among the states.
The various reinforcement learning algorithms described in the last two chapters rely on estimating state values, , or state–action values,
, directly.
The mean-square-error (MSE) problem of estimating a random variable from observations of another random variable
seeks a mapping
that solves
