Chapter 7 - Inference in Perception

Summary

If mental representations are important in problem solving, we need to provide a computational theory of how such representations are formed. Visual perception is the best place to start, considering how advanced research on visual representations is, when compared to other cognitive representations. Our physical world is three-dimensional (3D) and we see it as such, despite the fact that the sensory data input is a pair of 2D images on the back of the eye. Even if you close one eye, you still see the environment as 3D. It follows that the 3D percept is some kind of an educated guess (an inference) about the missing depth dimension. Formally, perception is an ill-posed inverse problem, whose solution requires a priori knowledge about the environment. What kind of a priori knowledge is both needed, and effective? It turns out that the symmetry of natural objects is both necessary and sufficient for making successful visual inferences. Combining sensory data with symmetry constraints leads to the veridical percepts of objects and scenes. In order to solve this visual problem optimally, the visual system finds the minimum of a cost function. The way the human mind solves this problem is completely analogous to how a least-action principle operates in physics. This analogy becomes important later when we discuss intuitive physics as a form of problem solving.