Perceptual learning in humans: An active, top-down-guided process

Abstract

Deep neural network (DNN) models of human-like vision are typically built by feeding blank slate DNN visual images as training data. However, the literature on human perception and perceptual learning suggests that developing DNNs that truly model human vision requires a shift in approach in which perception is not treated as a largely bottom-up process, but as an active, top-down-guided process.

Bowers et al. do the field a service with their thought-provoking commentary. If the problems currently characterizing deep neural network (DNN) models of human vision laid out in the target article are not adequately addressed, the field risks another winter. Bowers et al. sketch one important way forward: Building DNNs that can account for psychological data. I put forward that developing DNNs of human vision will foremost require a conceptual shift: From approaching perception as the outcome of a largely stimulus-driven process of feature detection and object recognition to treating perception as an active, top-down-guided process.

Building on the traditional notion of perception as a largely bottom-up process, mainstream computational cognitive neuroscience currently embraces the idea that simply feeding blank slate DNNs large amounts of training data will produce human-like vision. Yet, as Bowers et al.'s overview shows, this may not yet be the case. Based on the literature on human perceptual learning and action-oriented theories of perception, I contend that this may be a direct result from the manner in which DNNs are typically trained to “perceive”: In a passive, data-driven manner. This approach typically does not induce perceptual learning that generalizes to new stimuli or tasks in humans (Lu & Dosher, 2022), is very different from how babies learn to perceive (Emberson, 2017; Zaadnoordijk, Besold, & Cusack, 2022), and does not take into account the action-oriented nature of perception (Friston, 2009; Gibson, 2014; Hurley, 2001).

The most consistent finding in the literature on visual perceptual learning in human adults is that learning is highly specific to the trained stimuli and tasks (Lu & Dosher, 2022). For example, improvements are often not observed if the test stimulus has a different orientation or contrast than the trained stimulus or when the trained stimulus is relocated or rotated (Fahle, 2004; Fiorentini & Berardi, 1980). These findings indicate that the typical outside-in approach used in perceptual learning studies in which participants are presented with stimuli to detect or categorize tends to induce learning at too low levels in the processing hierarchy to support feature-, stimulus-, or view-independent learning. Indeed, more recent research suggests that transfer learning can be enhanced when learning can be top-down guided and connect to higher levels in the processing hierarchy (Tan, Wang, Sasaki, & Watanabe, 2019). For example, when the training procedure allowed for more abstract rule formation, complete transfer of learning between physically different stimuli was observed (Wang et al., 2016). These observations fit with recent findings that perceptual learning involves higher cognitive areas (Shibata, Sagi, & Watanabe, 2014; Zhang et al., 2010) and proposals that perceptual learning is a top-down-guided process (Ahissar & Hochstein, 2004). Perceptual development in infants is also more top-down guided than traditionally assumed (Emberson, 2017) and perception continues to develop through childhood based on acquired knowledge across a range of tasks (Milne et al., 2022). Yet the building of models of human vision still typically starts from the notion that human-like vision will simply arise by feeding blank slate DNNs many supervised training images, which may cause learning at too low levels in the processing hierarchy. Indeed, as Bowers et al. summarize, DNNs can be fooled by additive noise (Heaven, 2019), have difficulty generalizing learning to novel objects, and do not form transformation-tolerant object identity representations at higher layers (Xu & Vaziri-Pashkam, 2021). These problems conceivably reflect insufficient top-down-guided learning.

Research also shows that perception and action are interdependent processes, in particular during the development of perception (Zaadnoordijk et al., 2022). For example, kittens that are passively moved around, do not develop depth perception (Held & Hein, 1963), just like DNN's that are fed visual input do not perceive depth (Jacob, Pramod, Katti, & Arun, 2021). Humans are not passive perceivers, but continuously build on past experiences to actively predict and generate their own sensory information through action, thereby top-down driving their own learning (Boonstra & Slagter, 2019; Buzsáki, 2019; Friston, 2009; Gibson, 1988). That perception incorporates expectations about the sensory outcome of actions is demonstrated by the fact that humans who wear goggles that flip the visual field from left to right, do not perceive a normal world (albeit flipped left-right), but experience distorted perception (Kohler, 1963), caused by the disruption of normal sensorimotor contingencies. Moreover, recent studies show that responses in early visual cortex also reflect actions (Schneider, 2020). These findings cannot easily be explained by the classical view of the brain as processing information serially from sensory to cognitive to motor control stages (Hurley, 2001), each subserved by distinct brain regions – a view that currently still drives much of DNN research. Rather, they indicate that perception emerges from dynamic feedback relations between input and output, and does not merely entail the encoding of environmental statistics, but also the statistics of agent–environment interactions (Friston, 2010). Yet DNNs are generally trained in a passive way. This may cause shortcut learning or DNNs to latch onto features that do not matter to humans in categorizing objects. DNNs may focus on texture (Geirhos et al., 2022) or local rather than global shape (Baker, Lu, Erlikhman, & Kellman, 2018), because they never had to interact with objects, for which global shape knowledge is important. Notably, the development of global shape representations may depend on the dorsal stream (Ayzenberg & Behrmann, 2022).

To develop models of human-like vision, the field thus needs to turn the notion of perception on its head: From bottom-up driven to top-down guided and fundamentally serving agent–environment interactions. Important steps are already taken in this direction. For example, DNN architectures wired to top-down infer their sensory input has been shown to work at scale (Millidge, Salvatori, Song, Bogacz, & Lukasiewicz, 2022). There are also exciting developments in robotics, in which artificial systems equipped with the possibility to predict and generate their sensory information through action can top-down drive their own learning (Lanillos et al., 2021). DNNs have the potential to provide powerful ways to study the human brain and behavior, but this will require the incorporation of biologically realistic, action-oriented learning algorithms, grounding vision on interactions with the environment.