Response to commentaries on What Babies Know

Abstract

Twenty-five commentaries raise questions concerning the origins of knowledge, the interplay of iconic and propositional representations in mental life, the architecture of numerical and social cognition, the sources of uniquely human cognitive capacities, and the borders among core knowledge, perception, and thought. They also propose new methods, drawn from the vibrant, interdisciplinary cognitive sciences, for addressing these questions and deepening understanding of infant minds.

Thanks to all the commentators for reading my book and sharing their thoughts. I wish that I could engage with all the points made by each commentary individually, but space constraints require condensing. This response begins with preliminaries (sect. R1) and then addresses five questions: Does the book's account of the origins of knowledge hit the right balance between innate structure and learning (sect. R2)? Are there core knowledge systems that it neglected (sect. R3)? How does language relate to core knowledge, learning, and other symbol systems (sect. R4)? How does core knowledge interface with perception and thought (sect. R5)? Going forward, what kinds of research might best advance understanding of the origins, growth, and nature of human minds (sect. R6)?

R1. Preliminaries

I begin by correcting some confusions that I caused. As Schulz notes, the book's definition of innateness is inadequate, so here is a better one: A cognitive system is innate if it is present and functional on the infant's first effective informational [formerly, perceptual] encounters with the entities to which it applies. The corrected definition makes room for learning from text, television, or testimony and also for innate knowledge of entities for which prior perceptual encounters yielded no usable information. Schulz also notes an ambiguity in my argument for the abstractness of concepts with a long evolutionary history. I should have written that a core cognitive system that first emerged in distant ancestors and is present in their distantly related, contemporary descendants is more likely to center on abstract, broadly applicable concepts (like plant) than on more specific concepts (like coconut), because the former concepts will be useful for all descendant species, however much their environments differ.

Moore & Lewkowicz note that the précis contains no definition of learning. The book does (p. 71), but its definition is flawed by its focus on perceptual rather than informational encounters. A better definition considers a representational system to be learned if it is shaped by the learner's informational encounters with entities in its domain. Both revised definitions serve as invitations to research, especially research on controlled-reared animals and randomized experiments evaluating interventions to promote learning. In contrast, Moore & Lewkowicz's definition of learning, as “functional changes that result from experiences,” is too broad to guide research in developmental cognitive science: Falls can cause functional changes to cognition but concussions are not learned.

Finally, in the book's chapter on number, I characterized infants' number representations as approximate but took no stand on whether their imprecision inheres in the core number system or in the systems with which it interfaces. I was less careful in the précis and used the expression, “the approximate number system,” as if it were synonymous with the core number system. This usage is misleading, as research has not revealed whether the imprecision of numerical comparisons, made without counting, stems from limits to the core number system, to the perceptual systems that serve as its inputs, or to the systems for action planning, memory, and thought that support counting and other forms of exact enumeration. I will be more careful in this response.

R2. Core knowledge, nativism, and empiricism

Learning is impossible without cognitive systems that support it. For classical empiricists, human learning depended on innate sensory systems and mechanisms for learning by association. What babies know proposes a richer initial structure, with domain-specific cognitive systems providing abstract concepts that guide learning and reasoning throughout life. Many commentaries argue this account is too lean, and three argue it is too rich. In this section, I respond to two commentaries in the first category and all the commentaries in the second, deferring the remaining calls for more inborn structure to sections R3 and R5.

According to Margolis & Laurence, an abundance of innate cognitive capacities is needed to account for the development of knowledge in our species, including a universal grammar, representations of basic metaphysical distinctions, modal operators that support planning and counterfactual reasoning, representations of exact small numbers, and cognitive mechanisms that regulate attention and memory. Carey further argues that the core knowledge systems are limited by their iconicity; she calls for one or more innate representational systems that are propositional and language-like. I begin with one of the capacities that Margolis & Laurence call for – an innate modal operator representing possible states of affairs – and suggest that it builds on core knowledge.

Both in humans and in rats, place cells are activated during action planning, and research has focused on the planning process: When navigating rats arrive at an unexpected blockade in a familiar environment, they pause while place cells fire in sequences that correspond to different traversable paths to their goal. After generating representations of several paths, the animal chooses one and heads for the goal, as the précis describes.

Do the imagined place cell sequences represent possible routes to the navigator's goal? Consistent with that suggestion, distinctive neural activity accompanies the sequences that are generated before people or animals choose their next move. When rats pause at a choice point, the activation of place cell sequences occurs against a background of bursts of neural activity throughout the hippocampus, called sharp-wave ripples. In contrast, travel down real paths activates place cell sequences against a background of rhythmic theta waves. The coupling of sharp-wave ripples to the place cell sequences may allow animals to evaluate different action plans without creating false memories of travel on paths not taken. In humans, sharp-wave ripples also occur during recall of past events in episodic memory tasks (e.g., Norman et al., 2019), so they may serve to mark all simulated events that are detached from current reality, including but not limited to possible actions or states of affairs. If so, then further patterns of activity in the hippocampus or other brain structures might distinguish between representations of possible and past events.

These findings suggest, first, that representations of possibility indeed may be innate, as Margolis & Laurence argue, because they are evident in animals that have never taken any action (Farooq & Dragoi, 2019). Second, these representations build on core knowledge – the core place system, in this example – not on representations of a different kind. Third, contra Carey, the core place system is not wholly iconic, and representations of possibility are not wholly language-like: Both appear to straddle the divide between iconic and propositional representations, because each action plan has discrete components – representations of each place along the way – that together yield a holistic simulation of a course of action. However, action planning also requires more cognitive systems than the book discussed: In this example, one or more simulation engines, a process for searching through available options, and a process for distinguishing simulations from reality.

Three commentaries find the core knowledge hypothesis to be too strong. Reid claims that the capacities that I attribute to innate core knowledge can be explained, in part, by learning during the fetal period, in response to light sources outside the womb, but one consideration suggests only a limited role for visual learning during this period. Dark-reared rats and controlled-reared chicks perceive the world at birth at least as well as newborn human infants do, as Vallortigara described, but in most studies, they are confined to darkness prior to as well as after birth. Nevertheless, I salute Reid's efforts to bring cognitive research during the fetal period to bear on questions concerning the origins of knowledge. My hunch is that experience-dependent changes indeed shape the fetal brain, but the most important experiences are generated internally, by the systems of core knowledge.

Second, Adolph & Schmuckler reject the methods and conclusions of almost all the research discussed in the book. Our disagreements begin with their first sentence: “Inferences about infant perception and cognition must be based on their observable behaviors because babies can't talk, they don't follow instructions, and their physiological responses may reflect different psychological processes than adults.” With this dictum, they exclude all the methods and findings of research on animal behavior and in systems, cognitive and computational neuroscience. Their last phrase is true but argues for, rather than against, interdisciplinary research on the origins of knowledge. Functional brain imaging experiments reveal both common and diverging patterns of neural activity in humans, rodents, and chicks, as Vallortigara writes, and also in human infants, children, and adults. All these findings shed light on the origins and growth of human knowledge, as does work in many other fields, including philosophy, computer science, linguistics, and anthropology. The convergence in findings across these and other disciplines has been the single most exciting scientific development of my lifetime.

It is behavioral research in developmental psychology, considered on its own, that has sometimes suggested a misleading picture of infants' knowledge. For example, infants may careen down a sloping surface because they misperceive the slope; because they perceive the slope but don't know how to traverse it; because they know what actions are needed to traverse it but lack the strength or skill to perform them; or even because they know they lack these skills, as Gweon & Zhu or Berke & Jara-Ettinger might propose, and opt for trial-and-error learning. Decades of behavioral research, documenting infants' misadventures on slopes but committed to the position in the commentators' first sentence, have not distinguished these possibilities. The good news is that all these questions can be addressed by research in the interdisciplinary, developmental cognitive sciences.

I also dispute Adolph & Schmuckler's characterization of the arguments in the book as “rich interpretations based on lean ‘looking-time’ behaviors.” First, looking isn't lean: As Gibson taught us, it is an exploratory behavior, guided by what infants know and aimed at extending their knowledge. It is highly useful for behavioral research on infants, who are limited actors but eager explorers. Second, the book's conclusions depend on evidence that extends well beyond any single behavior: All the cited findings concerning object representations, for example, are corroborated by research using exploratory reaching as the outcome measure (e.g., Clifton, Rochat, Litovsky, & Perris, 1991; von Hofsten & Spelke, 1985). Moreover, many of its findings are supported by research on precocial animals, using locomotion and other actions as measures. Third, infants' knowledge is abstract but it isn't rich: Their object representations, for example, fail to distinguish a toy duck from a shoe.

Nevertheless, I am grateful to my academic siblings for introducing the concept of affordances into this discussion. I also treasure their account of Gibson's approach to their own work and fully endorse her reaction: You should do more experiments and figure out what is going on! Does blinking at an approaching object reflect infants' perception of impending or possible events, or is it a reflex, triggered by detection of patterns in the 2D optic array? Do infants topple down slopes because they misperceive them, misjudge the actions needed to traverse them, or revel in trial-and-error learning? Decades of research on infants, conducted with an exclusive focus on overt behavior, have not distinguished these possibilities. Multidisciplinary research could do so.

Finally, psychophysical experiments differ from other experiments not by the outcome measures they use but by the research strategy they follow: They leverage mathematical relationships between physical stimulation and perceptual experience to gain insight into perceptual mechanisms and processes. As a full-time faculty member, with her own infant laboratory, for only 3 years, Gibson lacked the time to extend her psychophysical studies of adults and children to infants. Soon after her mandatory retirement, however, Held, Teller, Banks, and others published studies using the simplest measures of selective looking – for example, did the baby look left or right? – in psychophysical experiments on visual development, and Keen (then Clifton), Werker, Kuhl, Aslin, and others used a related measure – head-turning – in psychophysical experiments on auditory localization and speech perception. Important insights will be lost if these rich literatures are rejected because of prejudices against the exploratory behaviors on which they rest.

Moore & Lewkowicz advance an analogy between the development of knowledge and the growth of organisms. I applaud their openness to insights from biology, but their use of this analogy obscures the specific (and considerable) challenges faced by every investigation into the nature, origins, and development of capacities to represent the world. Their claim that “…a developmental analysis must a priori consider all stimulation as potentially crucial unless its role has been empirically ruled out” sets an impossible standard for research: Because beating hearts are necessary for animal life, no studies on animals can remove all sources of stimulation. Instead, good experiments evaluate hypotheses against counterhypotheses that are reasonable, given the current state of knowledge. Setting a skewed standard, whereby empiricist claims are taken to be true unless proven false, introduces a bias that is especially harmful to empiricists, because experiments probing infants' learning become superfluous if their claims are deemed to be true in the absence of empirical support.

Nevertheless, the definitions and methodological standards that Moore & Lewkowicz's commentary articulates have been highly popular over the history of developmental psychology, and I thank them for giving me the chance to push back against them. The dismissal of all claims concerning the inherent, endogenously generated cognitive processes that give rise to innate knowledge, and guide learning, impeded the study of cognitive development for many decades. Fortunately, the interdisciplinary study of the origins and early growth of knowledge is now at an exciting time, as psychology, neurobiology, and other fields interact to discover how infants represent the world. I hope Moore & Lewkowicz enjoy the methods and insights that these synergies bring to our field.

R3. Are there more systems of core knowledge?

Nine commentators argue for additional core knowledge systems. The elegant experiments by Hespos & Rips show that infants distinguish nonsolid substances from objects. Their findings raise the possibility that infants have core knowledge of substances, but they also are compatible with the hypothesis that knowledge of substances is learned. Research could distinguish these accounts by probing knowledge of nonsolid substances in controlled-reared animals or newborn infants. If infants are endowed with core knowledge of substances, interesting questions arise concerning its content and its relation to the core object system, as Hespos & Rips note. Strictly speaking, objects do not pass through nonsolid substances, they displace them. Do infants expect the height of a liquid to rise, or the shape of a sandpile to change, when an object moves into the space that it occupies? Experiments on infants, animals, children, and adults, using multiple methods, could address these questions.

Duval argues for a system of place representation, focused on the appearances and affordances of landmarks, that is distinct from the core place system on which I focused. Indeed, animals and humans navigate by landmarks and use them to correct errors that will otherwise accumulate during path integration, as he notes. It is not clear whether the ability to navigate by landmarks arises from a domain-specific core knowledge system or from domain-general systems of perception and memory, but landmark-based navigation deserved more respect than my book gave it. Learning to recognize places by their distinctive landmarks is a key task for all navigators, and beautiful research illustrates how landmarks interplay with geometric place representations to enhance navigation performance, in creatures as distantly related as ants (Muller & Wehner, 1988) and human adults (e.g., Doeller & Burgess, 2008).

Drawing on the rich, subtle, and fascinating literature on infants' social reasoning, Hamlin and Tatone & Pomiechowska argue that infants have early emerging knowledge of social goals: Instrumental actions undertaken to benefit another agent. They point to studies in which younger infants selectively look or reach to characters who helped other characters without imitating them. In these studies, however, the reasons for infants' actions are not clear. First, infants may infer that such characters have helpful intentions, as Hamlin argues. Second, after seeing a protagonist repeatedly try and fail to achieve its goal, infants may view the character whose action completes the goal as a better agent, not a better social partner, because it is willing and able to move the action along. Third, infants may develop a social preference for characters who help others without understanding their acts of helping. Infants may deem such characters to be interesting, because they sometimes act on objects and sometimes engage with other characters, before they represent social actions like giving or helping.

Several considerations suggest that moral evaluations emerge through learning, rather than from an innate moral core (see also Hamlin, 2023; Spelke, 2023). First, infants' evaluations of helpers develop gradually in many experiments: For example, negative evaluations often emerge before positive ones. Moreover, as Hamlin notes, moral intuitions depend not on the acts we see but on actors' intentions: Pushing an agent up a hill is morally praiseworthy if the actor aims to fulfill the unsuccessful climber's goal, but not if she aims to push the climber out of her way. Because core knowledge builds on perception, it cannot readily distinguish between distinct construals of a single perceived action. Nevertheless, learning about predictive relationships between individual people's actions and their social behaviors may well influence infants' social evaluations, prior to the development of social agent concepts or true moral evaluation. Research could test these suggestions (see sect. R6).

Three commentaries argue for early emerging knowledge of triadic interactions involving the infant, a person who engages with him, and an object toward which the infant and his partner share attention. My book benefitted greatly from Grossmann's findings concerning young infants' brain responses to social others: His research provides evidence, I believe, for core knowledge of people as sentient beings who share their experiences in states of engagement. Does a further social distinction, between social engagements that are dyadic versus triadic, also emerge early in infancy? In the behavioral experiments he cites, young infants' longer looking when a person's gaze shifts back and forth between them and an object could occur for multiple reasons, because objects and hands are potent elicitors of attention. The cited neuroimaging experiments make a stronger case for core knowledge of triadic interactions, but their findings also could depend on the distinct, attention-eliciting effects of direct gaze, gaze shifts to objects, and object motion. These possibilities may be hard to distinguish using fNIRS, given its relatively low spatial and temporal resolution, but future neuroimaging experiments using other methods could test them (see Ellis and sect. R6).

Tauzin, Jacob, & Gergely argue that infants endow agents with pragmatic communicative intentions in the absence of language. Two of their cited studies focus on young infants but are open to multiple interpretations. First, infants may view face-to-face interactions as social engagements rather than communicative exchanges: A speaker's utterance may be interpreted as a social overture (like “hello”) and the listener's response, reproducing the speaker's action, may be viewed as social imitation (as in Powell & Spelke, 2013; although see Neff & Martin, 2023). Second, infants may learn, by 6 months, that a single speaker can be informative, efficient, or relevant to the current situation at different times within an episode, without integrating these three goals into a unitary communicative intention. These competing claims also should be testable, using methods that track the time course of signatures of action understanding and social engagement.

Kaufmann & Clément argue for an early-emerging system of intuitive sociology, focused on relations like shared group membership or dominance. I find this hypothesis highly worthy of testing. Kaufmann & Clément also note that learning about the social world bears a strong resemblance to learning about the navigable world. In both cases, the fundamental entities that populate these worlds are not geographical or social groups but individual places and people. Moreover, infants' learning of the specific social relationships that connect people resembles the learning, by navigating animals, of specific paths that connect places. This learning therefore could arise either from a core naïve sociology or from core knowledge of objects and places, together with a domain-general mechanism for representing the networks that connect distinct individuals within a domain (see sect. R5).

Finally, the commentaries by Berke & Jara-Ettinger and Gweon & Zhu point to an important gap in my book's coverage: Nowhere do I ask what babies know about themselves. This omission is not minor, because all the core systems apply to infants themselves. Infants cannot pass through walls or act on objects at a distance; they are living, agentive, and social beings; and their representations of number, geometry, and social relationships apply to themselves: Young children use their fingers in counting, retrieve information about their changing location as they navigate, and learn about relationships connecting them to other members of their social world (Thomas, Saxe, & Spelke, 2022). I am grateful to these commentators for signaling this gap in the book's coverage.

Conceptions of the self are hard to study: When the infants in Thomas' experiments see one of the two puppets comforting their mother, for example, do they learn something about themselves (I am close to this puppet), about the puppet (this puppet is close to me), about their family as a group (we are close), or all three? Berke & Jara-Ettinger suggest, however, that self-knowledge can be studied by focusing on situations that distinguish our experience of the world from the world itself, such as perceptual illusions, and Gweon & Zhu appeal to research by Stahl and Feigenson as evidence for infants' knowledge of their own violated expectations concerning the behavior of objects. Do these situations reveal unlearned representations of the self from the beginning?

Against that possibility, 1-year-old infants' exploration of objects that have behaved in surprising ways suggests that they are more focused on learning about the objects than on learning about themselves. When adults are presented with magic shows, we assume that we failed to detect the sleight of hand, and we may increase or redirect our attention during the next act. The infants in Stahl and Feigenson's studies don't respond in this manner, however; instead, they vary their actions on the object to test for its properties. They may do this for good reasons: Infants are great explorers and learners, and so their own minds are moving targets, given the rapid growth of their knowledge and skills. Thus, I lean toward the view that explicit knowledge of the self is learned, and it grows as infants discover their own, and other people's, diverse and changeable perspectives on the world. But this topic deserved discussion in What babies know, and I'm inspired, for the sequel, to think more about it. As long-standing research by Rochat (2018) and others has shown, implicit self-knowledge either begins to be learned in infancy or is present from the beginning.

R4. Core knowledge and language

Goldin-Meadow appeals to research on deaf children of hearing parents, who communicate by homesign, to challenge the book's claims for language learning as the primary process that carries infants beyond core knowledge and toward richer conceptions of people as social agents. Although I took no stand on how infants learn their language, studies of homesign accord with Carey's suggestion that language acquisition depends on a seventh domain-specific system of core knowledge. If homesigners' invented language is accompanied by the emergence of knowledge of individual actions as simultaneously social and object-directed, like gift-giving, then Goldin-Meadow wins, and language attainment, not language learning, underlies infants' concepts of social actions.

However, I argue (in ch. 10) that children's fast and flexible cultural learning depends on learning of an established conventional language, because such languages are shaped by generations of speakers who have aimed to communicate efficiency, informatively, and relevantly. In such a language, words that convey the culture-specific concepts that provide the most useful perspectives on the world will tend to be short and frequent, and general learning mechanisms, biased to learn from simpler and more frequent events, will orient children toward their culture's most useful concepts in ways that language invention cannot. I predict, therefore, that homesigning children will be slower, less effective cultural learners than are children who learn an established conventional language. Consistent with this prediction, some studies of homesigning adults, and of adult speakers of emerging sign languages, provide evidence for limited mastery of culturally variable concepts of exact numbers (Spaepen, Coppola, Spelke, Carey, & Goldin-Meadow, 2011, 2013), spatial relationships (Pyers, Shusterman, Senghas, Spelke, & Emmory, 2010), and mental states (Pyers & Senghas, 2009).

My chapters on language and the construction of new concepts appealed to Waxman's research with older infants and toddlers, showing that language allows them to characterize the same perceived entity in diverse ways: For example, as a car, a Fiat, or a blue thing. She also finds effects of language on 3-month-old infants' categorization of diverse pictures of dinosaurs, but this finding is open to multiple interpretations, as she notes. For example, speech and other vocalizations, like lemur calls, may alert infants to the presence of something worth attending to, allowing focused perceptual analysis of object properties. I hope future research will chart the emergence of sensitivity to the different perspectives on objects that language can convey. I hypothesized, in chapter 10, that infants begin to distinguish these perspectives at about 12 months, when they first come to view others' mental states as both phenomenal and intentional. Consistent with this hypothesis, word learning accelerates and becomes more confident in the first months of the second year (Bergelson, 2020).

Against my account of core knowledge and language as the sources of our uniquely human concepts and cognitive skills, Schulz argues that talents for cultural learning and symbol use are the foundations of uniquely human cognitive accomplishments. I agree that these talents are key signatures of distinctively human intelligence, but what inherent talents underlie them? Young children readily learn their native language, but other symbol systems, including pictures, toy cars, maps, and graphs, are harder for children to master (DeLoache, 2004), unless the symbols are accompanied by language (Winkler-Rhoades, Carey, & Spelke, 2013). Humans learn diverse symbol systems, I suggest, because we are endowed with core knowledge and with an inherent attunement to the symbolic functions of language.

Margolis & Laurence pushed back against my argument that language learning reverses the curse of a compositional mind and claim there is no such curse: The richer the child's innate conceptual repertoire, the easier concept learning will be. I do not claim that possession of a combinatorial system that generates all humanly attainable concepts will make concept learning impossible in the absence of language learning: A machine with infinite time will find every concept in it. Concept learning will be slow if concepts are drawn from a rich mental language in a culture with no public language, however, because if they are sampled in a manner that allows learning in any culture, then children will need to search through a vast space of combinations to find the right ones. In contrast, finding the right concepts in a vast repertoire becomes more manageable if concept learning is informed both by core knowledge, which applies to all cultures and environments, and by speech in the conventional language of one's culture, which provides multiple cues to the concepts that its members find most useful.

In the book, I argued that core knowledge is never expressed in language, because it is universal (and therefore can be left unsaid) and unconscious (and therefore cannot be explicitly accessed). Contrary to the second claim, Lin & Dillon's elegant experiments suggest that ordinary language activates representations from core knowledge: When two connected line segments are described as an incomplete object, people connect them; when described as an incomplete path or abstract pattern, they extrapolate the pattern. As they note, it isn't clear whether these verbal descriptions activate core knowledge directly or indirectly, by activating explicit concepts that build on core knowledge. In either case, however, this activation may explain why games that connect images of sets or forms to numerical and geometric language and symbols have stronger, more enduring effects on children's math learning than does play with language and symbols alone (Dean, Dillon, Duflo, Kannan, & Spelke, 2023; Dillon, Kannan, Dean, Spelke, & Duflo, 2017). I look forward to research probing the links connecting core knowledge to language and endorse Lin & Dillon's conclusions concerning its potential implications for education, economics, and the developmental cognitive sciences.

Finally, language learning may address an important problem raised by Kaufmann & Clément, who appropriately distinguish between concepts of individuals, like a particular place, object, plant, or animal, and second-order abstract properties like number and geometry. The latter concepts are radically ambiguous: “How many things are in this room?” is a question with different answers, depending on what things one counts. Solving this problem, they suggest, requires either that different core systems form a multilayered hierarchy, or that the core systems of number and geometry aren't as abstract as my discussion in What babies know implies. I favor the second option: The core systems of number and geometry are limited. Although they represent numerical magnitudes and geometric shapes that can be transformed by operations of arithmetic and geometry, and they support children's learning of, and adults' reasoning about, mathematics, they lack the power of explicit mathematical concepts.

The core number system differs from our most intuitive explicit number system – the integers – in at least two ways. First, it fails to support exact enumeration, either because it is inherently approximate or because it interfaces with systems of limited resolution. Second, because core knowledge systems compete for attention, infants can activate representations of objects or numerical magnitudes but not both at once, so the core number system fails to capture the hierarchical structure that allows for representations of three people, three animals, or three shoes. Experiments with navigating animals, and with human adults performing a virtual navigation task, reveal a similar failure to capture, in a unitary representation, the positions of environmental boundaries and of landmark objects. Our core systems of number and geometry therefore do not operate in accord with the hierarchical organization of our mature mathematical concepts. Language, however, is a hierarchically structured, symbolic system, so it might provide a medium for constructing mature representations of number and geometry: topics for my next book.

R5. Core knowledge, perception, and thought

If my aim, in writing What babies know, were reduced to a single sentence, it would be this: To make the case for a level of representation beyond perception and thought, and for the critical contributions of domain-specific cognitive systems at this level to the origins, growth, and nature of human knowledge. Many commentators are not convinced, and for a good reason: The book did not devote nearly enough space to cognitive systems and processes at the borders between core knowledge, perceiving, and thinking.

Carey argues that both perceptual and core knowledge representations are iconic, or image-like, whereas conceptual representations are discursive, or propositional and language-like. She concludes that cognitive systems come in two rather than three kinds: perceptual and conceptual systems. I dispute these claims. First, the best theories of perception, dating back to Helmholtz, appeal to representational systems that are both iconic and propositional, as they depend on unconscious inferences concerning the sources of sensory experience. Similarly, the contemporary theories of vision described in the book and précis appeal to graphics engines: Generative models, written in an internal programming language, that produce holistic 2D images of the light-reflecting surfaces in a 3D scene. If these theories are correct, then perceptual systems combine iconic and discursive representations (see also Quilty-Dunn, Porot, & Mandelbaum, 2023).

Following Ullman, Spelke, Battaglia, and Tenenbaum (2017), I suggested that core knowledge systems also function as generative models of the entities in their domains: Models that support learning when they are run in the forward direction and that support inference both when run forward and when inverted. If that is true, then the representations formed by core knowledge systems also are both iconic and discursive, contrary to the dichotomy urged by Carey and by Block (2022). Finally, investigators from diverse perspectives have long argued that explicit thought, memory, and prospection involve both mental simulation and language-like reasoning (see Kosslyn, 1980; Shepard, 1984). This argument has been made by investigators with diverse views on the nature of the representations that guide reasoning and simulation, but one family of accounts appeals to the same sorts of generative models as those proposed to underlie innate capacities for visual perception and core knowledge (e.g., Tenenbaum, Kemp, Griffiths, & Goodman, 2011; Ullman, 2015; Ullman & Tenenbaum, 2020).

In sum, perception, thought, and core knowledge all may depend on image-like simulations produced by language-like representations and computations. What, then, distinguishes core knowledge from perception and thought? First, our perceptions and beliefs change with experience: Older children are better at distinguishing faces and recognizing known others than younger ones, and chess masters see relationships on a chess board and plan moves that novices miss, as Krøjgaard, Sonne, & Kingo (Krøjgaard et al.) notes. Core knowledge, by contrast, shows the same signature limits in adults and infants. Second, perception and belief depend in part on unconscious processes but give rise to conscious experience, whereas core knowledge is wholly unconscious: We can become aware of its existence and properties by doing experiments but not by introspecting. Third, as Vallortigara's commentary argues, the plastic neocortex is the seat of our conscious, malleable perceptions, memories, planning, concepts, and beliefs, whereas core knowledge in animals and infants likely depends on subcortical brain systems that are hard-wired and impenetrable.

Scholl raises crucial questions concerning the relationship between core knowledge and visual perception of objects, scenes, causal relations, and animacy. For decades, he and his associates have built on classic experiments – such as Michotte's studies of the perception of causality and Heider and Simmel's study of perception of animacy – to probe the processes by which we perceive these phenomena. Although perception itself is conscious, his studies reveal that many perceptual phenomena depend on processes that are unconscious and automatic. Moreover, the representations that they deliver are strikingly similar to those found in studies of infants, even though the experiments themselves differ. Scholl's commentary provides the simplest, most natural explanation for this consilience: Core knowledge systems are perceptual systems.

Although Scholl notes that the consilience is not complete, it may be even stronger than his commentary argues, as he focuses on wonderful findings from foraging and predator-detection tasks but no findings bearing on perceptual knowledge of places or people. The occipital place area (OPA) of the visual cortex is consistently activated by visual scenes, it is specifically attuned to the navigational affordances of these scenes, such as an open doorway in a room (Bonner & Epstein, 2017), and it is causally involved in human reorientation (Julian, Ryan, Hamilton, & Epstein, 2016). Studies of other regions in the occipital lobe connect mid- or high-level visual perception to core social cognition: Not only to representations of agents, as Scholl described, but also to representations of individual people and their relationships: crucial information for navigating the social world. Although Scholl notes that no visual area specializes in representing acts such as helping or gift-giving, earlier-emerging aspects of social knowledge may well have counterparts in the multiple visual areas that are sensitive to faces, as Carey describes.

There are two ways to think about these findings. First, my book may be misnamed: All the abilities that I find in infants may depend on perceptual systems, not systems of knowledge. It is hard to construe navigation or social reasoning as perceptual abilities, however, for several reasons. First, navigation builds on enduring representations of unseen destinations and of the paths that led from one place to another in the past, and social reasoning builds on enduring representations of the imperceptible bonds that connect one person to another. We may learn how two people are related by observing their interactions, but we store this information in a way that respects their previously experienced social relationships. Second, both these tasks are associated with activity in the hippocampus: A structure that is commonly associated with memory and action planning and is rarely considered to be a perceptual system. For these reasons, I favor the other possibility that Scholl's discussion invites: Core knowledge lies between or beneath our perceptions, memories, beliefs, and plans. If this view is correct, then perceptual signatures of core concepts, like the retinotopic adaptation effects of repeated viewing of causal interactions, would not be explained by adaptation of the core knowledge system but by adaptation of the cortical, perceptual representations that it activates.

If that's true, then the processes occurring at the interfaces of core knowledge with perception and thought beg for further study, and the commentary by Kaicher, Conti, Dedhe, Aulet, & Cantlon (Kaicher et al.) suggests ways to study them. Kaicher et al. argue for cognitive systems with all the properties of core knowledge systems except two: The systems crosscut the core domains on which the book focused, and they contribute to the efficiency and adeptness with which we perceive and reason, rather than to the content of knowledge. For example, the system that gives rise to categorical perception applies to entities in diverse domains, but it operates automatically and unconsciously, as do the systems of core knowledge.

I think there are good reasons to distinguish domain-specific systems of knowledge from domain-general systems for optimizing cognitive processing, but I agree that both types of systems are needed to explain how minds work, at all ages. Categorical perception reflects a fundamental property of perceptual systems and perceptual learning; it therefore serves to foster both the rapid identification of the entities in each core domain as well as abilities to distinguish one such entity from another and to learn about each entity's properties and behavior: Crucial skills for navigating the geographical world, as Duval argued, the social world of people and their relationships, as Kaufmann & Clément argue, and the world of object kinds, with distinctive forms and functions, as Liu & Xu's research reveals. By increasing the efficiency of perceptual processing, categorical perception enhances learning at the interface of perception with core knowledge.

I agree that the development of knowledge requires more cognitive mechanisms at the interface between core knowledge and thought. Krøjgaard et al. usefully propose that mental rotation, episodic memory, and chess playing depend on such systems. In section R2, I argued that cognitive mechanisms for simulating events (including rotations of objects), and for distinguishing simulations from reality, are needed to account for core knowledge, perception, memory, and reasoning. Episodic memory depends on the hippocampus and likely is activated and strengthened when humans or animals review past events. Chess playing requires both perceptual mechanisms permitting rapid analysis of relationships between the pieces on the board, and mechanisms of action planning allowing comparisons among sequences of possible moves. These processes operate at the interfaces of core knowledge with perception and with memory, prediction, and planning.

These considerations suggest an alternative account of the findings with which Jenkin & Markson and Liu & Xu challenge two of my central claims for core knowledge: The claim that core knowledge is constant over development and unaffected by later-emerging beliefs or attitudes, and the claim that the core knowledge systems form a natural kind, with common functions and modes of operation. Against both claims, Jenkin & Markson argue that the core social system is revisable, and Liu & Xu argue that all the core systems are revisable, based on their findings that preschool children develop beliefs at odds with core knowledge of objects and agents when given verbal questions about events that are similar to those providing evidence for core knowledge in infants. Liu & Xu also argue that the object and number systems are harder to revise, as are perceptual systems, whereas the agent and social systems are more readily revised, as are belief systems. These arguments cut to the heart of the claims made in What babies know.

Adults' and children's explicit beliefs about the entities singled out by core knowledge, and the actions that we perform on the entities in core domains, are various, changeable, and sometimes at odds with core knowledge. Nevertheless, the hypothesis of constant core systems, interfacing with malleable systems for perceiving and thinking, seems far more plausible to me than the hypothesis that core knowledge itself is revisable. For example, mathematicians develop concepts of complex numbers and high-dimensional spaces. Where research has been conducted, however, their reasoning about difficult problems in mathematics has activated the same systems of core knowledge as are activated by numerical or geometric tasks in children and ordinary adults (e.g., Amalric & Dehaene, 2016).

Thus, I differ from Liu & Xu over the sources of our malleable, flexible reasoning. Infants reason flexibly, I submit, because the properties captured by each impenetrable core knowledge system apply to all habitable natural environments. For example, the core agent system applies to all actions that agents can undertake: Not just reaching for objects and locomoting to places, but reaching to places and locomoting to objects. That is why the 3-month-old infants in the study cited by Liu & Xu learned with equal facility that the goal of a reach was an object or a place, depending on the evidence that they received. The flexibility of children's learning about agents and their goals is consistent, moreover, with the impenetrability of core knowledge: Impenetrable core knowledge systems can promote flexible learning throughout life, because the abstract properties they capture apply to all the entities in their domain, for people of all ages, in all environments. Humans go beyond core knowledge, and even contradict it, when we develop explicit, learned beliefs, because beliefs are malleable. Core knowledge is not malleable, however, and so it continues to function, despite these beliefs.

Jenkin & Markson's commentary, focused on the core social system, raises a further question about the interface of this system with action and thought: How and why do children come to organize their social knowledge in accord with social categories of people, based on attributes like race, gender, or social class, developing biases toward or against individuals in these categories, if the core social system applies to all potential social beings and supports learning about individuals of all races (Kinzler & Spelke, 2011) and, indeed of many species (e.g., Pascalis, de Haan, & Nelson, 2002)? I believe these effects stem from the mechanism of categorical perception discussed by Kaicher et al.: As children gain increasing exposure to social beings of a familiar race, they come to perceive faces of that race more clearly, and information conveyed by the same-race face comes to them more vividly and rapidly: They quickly see, for example, that a pictured face of a familiar race looks happy or scared, whereas the emotion expressed by a pictured face of a less familiar race will be seen more slowly and less vividly. Later in development, responses to individuals of differing races may be modulated by explicit beliefs about the characteristic attributes and behaviors of people in different groups. At no time, I believe, will core knowledge of agents or social beings change. Instead, changes in racial attitudes likely depend on the plasticity of perception and thought. These are largely untested predictions, however. In the final section, I ask how future research might serve to shed more light on the development of knowledge in infancy.

R6. Beyond What babies know

What babies know omitted important questions. Throughout my research on infants' knowledge of number, for example, I wondered whether the core number system was exact or approximate, but I concluded that the question was not answerable by current methods: An inability to distinguish six from seven could stem from limits either to the core number system or to the perceptual, memory, and action systems with which it interfaces. The lack of evidence bearing on this question was not a reason to avoid discussion of the question, however: On the contrary, such discussion is a needed prelude to research. A second omission occurred in chapter 10, where I proposed (a) that infants' language learning brings them a new concept of people as social agents, based on evidence for the development of new concepts of social actions like helping and gift-giving, and (b) that in the absence of language learning, domain-general learning processes support a weaker understanding of social agents as beings whose specific actions on objects may predictably precede or follow specific social gestures. I stand by these claims but regret my failure to consider how language learning, and language-independent predictive learning, might combine to account for developmental changes in social cognition, through processes occurring at the border between core knowledge and thought.

In general, the commentaries have prompted me to think more about the interfaces between perception, thought, and core knowledge. At many points in the book, I suggested that neuroimaging studies of human infants could shed light on open questions about the origins of knowledge in infancy, but I never mentioned experiments using the methods of multivariate pattern analysis (MVPA), including representational similarity analysis (RSA). Although other ways of analyzing brain data have shed light on diverse aspects of numerical representations, some questions have proved exceedingly hard to answer, despite a rich body of research using the methods of psychophysics and cognitive neuroscience on adults, children, and infants. Ellis focused on one such question: Do infants, children, and adults represent number per se, or do they represent continuous variables that correlate with number, such as continuous spatial extent or temporal duration?

Ellis proposed that neuroimaging experiments using MVPA in general, and RSA in particular, can address this question. Indeed, a recently published study, conducted by Dehaene-Lambertz and her collaborators, embraced this challenge. Gennari, Dehaene, Valera, and Dehaene-Lambertz (2023) used MVPA to decode for number in 3-month-old infants, using high-density electroencephalography (EEG). As infants rested or slept, they heard sequences of tones varying in number (4 or 12), sequence duration, tone duration, and also tone frequency and timbre. Gennari et al. trained a decoder to identify and distinguish between sequences of 4 and 12 tones, using input from 256 sensors on the baby's scalp during brief intervals that followed the end of each sequence. During training, the decoder was presented with a critical subset of these intervals, chosen such that successful discrimination of the two numbers required that it ignores differences in sequence duration, tone duration, and the other variables. After training, the decoder reliably distinguished 4- from 12-tone sequences in the remaining data, providing evidence for representations of number in the infant brain.

Using RSA, moreover, Gennari et al. found that the response to number that was trained on the auditory sequences, heard during sleep, generalized with no further training to stationary visual arrays that the infants viewed while awake, before or after the sleep session. The latter finding accorded with earlier behavioral research on newborn infants (Izard, Sann, Spelke, & Streri, 2009) but went beyond it, because in this study, tones of different numbers were randomly intermixed, engendering no expectations that either number would be repeated, and infants were tested while drowsy or sleeping, allowing tests for generalization not only over changes in modality but also over changes in the infant's state.

Further analyses by Gennari et al. speak on Ellis' discussion of the vexed question of whether infants respond to number, or to continuous variables that correlate with number. They showed that the infants' brain signals allowed not only for successful decoding of number in the tone sequences but also for successful decoding of sequence duration and tone duration. Moreover, their analyses showed that infants' brain responses to the two duration variables were independent of their brain responses to number. Thus, the experiment provided evidence that 3-month-old infants represent number and two aspects of duration. It is hard to imagine how the independence and robustness of infants' numerical representations could have been tested without these methods and analyses. Using MVPA and RSA, Gennari et al. discovered a signal in the infant brain that is specific to number, as well as signals specific to other quantities.

Might similar experiments resolve the question of whether the core number system delivers an approximate or exact representation of number? Further RSA analyses by Gennari et al. showed that a decoder, trained on the interval that followed the third and the seventh tone in the 12-tone sequences, came to distinguish between them, even though the sleeping infants never heard sequences consisting of three or seven tones. This finding suggests that infants increment their numerical representations after each tone in a sequence, raising the possibility of exact numerical representations at each step in the sequence. Against this conclusion, the decoder failed to discriminate the third from the fifth tone, but this failure could have occurred because the 4- and 12-tone sequences that the infants heard all differed by a large ratio, encouraging a focus on approximate numerical magnitudes. I hope future studies will use these methods to test for exact number representations in infants. If the core number system is exact and error is introduced by perceptual and memory systems, then the brains of infants, presented in sleep with sequences of five versus six tones, for example, might produce a signal that a coder could be trained to detect, independently not only of continuous variables but also of the signals generated by other brain systems.

If decoders can use data from functional brain imaging to decode for numerical representations, then they might also serve to decode for representations of social actions and of the people who engage in them. Using these methods, investigators could focus on the changes in brain activity that occur with changes in children's understanding of social and communicative actions, and of the mental states of the people who perform such actions. As babies begin to learn that people's social gestures and object-directed actions occur in regular patterns, does this learning change their conceptions of people as social agents, or are concepts of people invariant over development, with changes only to infants' understanding of the actions they perform? Neural recordings, analyzed by MVPA and RSA, might address these questions.

Another potential avenue for advancing understanding of cognition in infancy is suggested by Vallortigara's commentary. He noted that newly hatched chicks, which respond to objects, places, and agents similarly to human infants, represent objects primarily via a midbrain structure, the optic tectum, that is a homologue of the human superior colliculus, which is also a subcortical brain structure. In the book, I speculated that all the core knowledge systems reside in subcortical structures, from which their activity (prenatal and postnatal) propagates to the plastic cortical regions underlying perception, memory, and learning. To my knowledge, no one has imaged subcortical activity in chicks or human infants during performance of tasks providing evidence for core knowledge. Such studies might shed light on the operation of the core knowledge systems and their constancy versus malleability by experience. Further studies of the activity that propagates from subcortical systems to the cortex then could address open questions concerning the interface between core knowledge, perception, and thought.

Beyond research in cognitive neuroscience, advances promise to come from field research, conducted in diverse countries and cultures, and following infants over extended timespans, as Lin & Dillon discussed. Still more advances may come from studies of infants whose everyday experiences differ from those of the infants who most often are studied by developmental cognitive scientists: For example, infants who learn their language from overheard speech, because adults in their culture do not speak to infants; infants with limited vision; infants with exceptional abilities like absolute pitch; or deaf children who invent their own language, as Goldin-Meadow argued. Finally, I look forward to insights from computational cognitive science, leveraging the data from experiments testing large samples of infants recruited through online platforms, large-scale field experiments, or collaborative replications of classic findings. Analyses of data from these sources could serve to evaluate diverse computational models of infant cognition and learning (e.g., Gandhi, Stojnik, Lake, & Dillon, 2021), including the probabilistic generative models discussed in the book. What babies know featured few experiments on infants using any of these methods, leaving rich territory for future books to explore.