Investigating infant knowledge with representational similarity analysis

Abstract

Decades of research have pushed us closer to understanding what babies know. However, a powerful approach – representational similarity analysis (RSA) – is underused in developmental research. I discuss the strengths of this approach and what it can tell us about infant conceptual knowledge. As a case study, I focus on numerosity as a domain where RSA can make unique progress.

“What Babies Know” (Spelke, 2022) is a state-of-the-art, inclusive, and thoughtful survey of what babies know about a few key domains, making the book a must-read for junior scientists in the field. Perhaps most striking is the breadth of research methods and model systems used in the experiments covered: From various behavioral approaches, to neuroimaging, to computational models, to animal neurophysiology. Even so, this commentary will highlight a method mostly absent from the text (and field) that could offer unique insight into infants' knowledge: Representational similarity analysis (RSA; Kriegeskorte, Mur, & Bandettini, 2008). I will discuss this method and explain its strengths, then focus on numerosity as a case study. Specifically, I will consider ways to address questions that the target book highlights as unresolved.

RSA is a method for evaluating the degree of representational similarity between exemplars in a set. RSA, along with the more straightforward “pattern similarity” method (Carlson, Schrater, & He, 2003), has been used frequently in vision science to compare the representations of exemplars, like faces, scenes, and objects. RSA is most often performed with functional magnetic resonance imaging (fMRI) (i.e., comparing the patterns of activity across voxels that exemplars evoke) but can be applied to any data where metric differences can be measured between exemplars, including electroencephalography (EEG; Xie et al., 2022) and behavior (Spriet, Abassi, Hochmann, & Papeo, 2022). One example of a question that RSA can ask is whether the representation of exemplars cluster according to their perceptual similarity (i.e., exemplars that are perceptually different have low similarity), or conceptual similarity (i.e., exemplars that are conceptually different have low similarity). Analyses of this type show that different parts of the brain can divide exemplars either perceptually or conceptually (Bankson, Hebart, Groen, & Baker, 2018).

A key benefit of RSA, and what makes it distinct from many popular methods in developmental science, is that it can test both the degree of difference between representations and the degree of similarity. By way of contrast, a habituation paradigm can test when infants think one exemplar (i.e., the habituation stimulus) is different from another (i.e., the test stimulus). However, suppose infants do not dishabituate to the test stimulus: That does not necessitate infants think the habituation and test exemplars are similar, but instead may result from poor study design (e.g., exposure is too short) or low statistical power. By contrast, high similarity in RSA typically (although not always, e.g., Spriet et al., 2022) entails positive evidence of similarity. For instance, in RSA with neuroimaging, high similarity means that neural patterns have a high correlation, which should not be expected by chance. An additional benefit of RSA arises when it uses neuroimaging data: It can disentangle simultaneous representations (Bankson et al., 2018), as illustrated in the example above where conceptual and perceptual information were localized in different brain regions.

Despite this potential value, RSA remains an uncommon tool in infant research. A few exceptions exist: Spriet et al. (2022) recently used RSA to understand the clustering of visual stimulus representations based on gaze behavior. Moreover, Xie et al. (2022) used EEG to measure neural responses to categories and evaluated their clustering. Studies using fMRI are surprisingly absent from this list, but this is likely to change soon given the recent success of awake infant fMRI (Ellis et al., 2020, 2021; Kosakowski et al., 2022; Yates et al., 2022). The aforementioned articles use RSA to address questions about visual perception, but RSA is flexible enough to tackle broad questions about what infants know. In what follows, I consider how RSA can resolve lingering questions in infant's knowledge of numerosity.

Over the last 30 years, research has led to incredible progress in understanding what infants know about number. Nonetheless, there are at least two questions that RSA can advance. The first question is whether numerosity is represented conceptually, rather than reflecting mere magnitude of perceptual content. Both historically (Clearfield & Mix, 1999) and recently (Leibovich, Katzin, Harel, & Henik, 2017), researchers have argued that attempts to de-confound perceptual magnitude from conceptual number have been insufficient. For instance, many experimental controls for perceptual confounds have prioritized the wrong stimulus properties (Yousif & Keil, 2021). A second question RSA can tackle is the extent to which a signature of the numerosity system – namely, that differences in number depend on a ratio (AKA ratio dependence) – is consistent across the number line. Ratio dependence means that arrays containing 4 and 8 dots are perceived as just as different as arrays containing 8 and 16 dots. There is compelling evidence that infants represent numbers according to a ratio for values greater than 3 (i.e., above the subitizing range), but it remains unclear whether quantities lower than four are processed with ratio dependence (Hyde & Spelke, 2011; McCrink & Wynn, 2004; Starr, Libertus, & Brannon, 2013; Uller, Carey, Huntley-Fenner, & Klatt, 1999; see the target book for an extended discussion on this topic). These two questions can be addressed in the following RSA study.

Infants would see arrays of dots while undergoing fMRI (EEG, MEG or functional near-infrared spectroscopy [fNIRS] would be a viable substitute for some of these analyses). Across trials, arrays will differ in the number of dots from 1, 2, 3, 4, 8, and 16. These numbers include the subitizing range (1–3) and beyond (4–16). The dots will be presented in one of the two sizes: (1) the dots are all the same size, regardless of the quantity of dots in the array, and (2) the dots are scaled so that they span the same total size (where size is based on additive area; Yousif & Keil, 2021). The pattern of activity in different brain regions that are evoked by each array, averaged across repetitions, will be compared to all other array types to complete the RSA.

With this relatively simple design, four distinct analyses can assess infant knowledge of number:

1. (1) Nonnumeric magnitude: are arrays with the same area (i.e., nonnumeric magnitude) more similar to each other than arrays with different areas, even when the number of dots differ? Brain regions sensitive to this nonnumeric magnitude will likely include both sensory systems and regions that support number processing in adults (e.g., the parietal cortex), as shown previously in adults (Sokolowski, Fias, Ononye, & Ansari, 2017).

2. (2) Numeric magnitude: are arrays with the same number of dots more similar to each other than arrays with different numbers, even when the size of the dots differs? Numeric magnitude has been shown to recruit the parietal cortex in infants (Hyde, Boas, Blair, & Carey, 2010). In adults, neural regions that code for numeric and nonnumeric magnitudes overlap, but only partially (Sokolowski et al., 2017). Using fMRI with awake infants, it's possible to test the degree to which these computations are supported by different systems: a viable hypothesis is that they start out similar and diverge during development.

3. (3) Ordinal number representations: akin to Lyons, Ansari, and Beilock (2015), are numbers represented ordinally (e.g., 2 is more similar to 3 than it is to 4)? This is particularly interesting for quantities in the subitizing range, where the processing of numeric magnitude may be served by object tracking (Uller et al., 1999); thus, representations may not be ordinal in this range.

4. (4) Ratio dependence: does representational similarity correspond more to absolute numerical differences (e.g., 2 is equally similar to 1 and 3) or a ratio difference (e.g., 2 is more similar to 3 than it is to 1)? Ratio dependence is a key indicator of a magnitude-estimation system (Meck & Church, 1983) and has been found for values in the approximate number range in infants (Xu, 2003). Whether ratio dependence exists for quantities in the subitizing range remains unclear (Hyde & Spelke, 2011; although see Starr et al., 2013).

These analyses highlight the flexibility of RSA and neuroimaging: It can test questions about the neural substrate of infant cognition (e.g., is the neural implementation of numerosity continuous across development) and also the nature of cognitive representations (e.g., what is the relationship between the representations of quantities). In this case study, neuroimaging gives both confirmatory and unique answers to questions regarding infant numerical perception. Even more exciting is that this is just a taste of what RSA and neuroimaging can do to help us understand the infant mind.