8 - Lending a helping hand to hearing: another motor theory of speech perception

Summary

… any comprehensive account of how speech is perceived should encompass audiovisual speech perception. The ability to see as well as hear has to be integral to the design, not merely a retro-fitted after-thought.

Summerfield (1987)

The “lack of invariance problem” and multisensory speech perception

In speech there is a many-to-many mapping between acoustic patterns and phonetic categories. That is, similar acoustic properties can be assigned to different phonetic categories or quite distinct acoustic properties can be assigned to the same linguistic category. Attempting to solve this “lack of invariance problem” has framed much of the theoretical debate in speech research over the years. Indeed, most theories may be characterized as to how they deal with this “problem.” Nonetheless, there is little evidence for even a single invariant acoustic property that uniquely identifies phonetic features and that is used by listeners (though see Blumstein and Stevens, 1981; Stevens and Blumstein, 1981).

Phonetic constancy can be achieved in spite of this lack of invariance by viewing speech perception as an active process (Nusbaum and Magnuson, 1997). Active processing models like the one to be described here derive from Helmholtz who described visual perception as a process of “unconscious inference” (see Hatfield, 2002). That is, visual perception is the result of forming and testing hypotheses about the inherently ambiguous information available to the retina.