Binz et al. argue compellingly for meta-learning as a tool to understand adult cognition, but their vision is incomplete: Meta-learned models are particularly apt tools for studying development. As the authors note, meta-learned models provide a natural foundation for theorizing about learning to learn (Wang, Reference Wang2021) and for understanding experience-driven changes in goal-directed behavior (Nussenbaum & Hartley, Reference Nussenbaum and Hartleyin press). Here, however, we consider the authors’ central argument and focus not on meta-learning algorithms or the process of “learning to learn,” but rather on meta-learned models as a tool for understanding developmental changes in learning. To date, research has revealed age-related shifts in the algorithms and neural circuitry that underlie learning (Bolenz, Reiter, & Eppinger, Reference Bolenz, Reiter and Eppinger2017; Gualtieri & Finn, Reference Gualtieri and Finn2022; Hartley, Nussenbaum, & Cohen, Reference Hartley, Nussenbaum and Cohen2021; Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019; Raab & Hartley, Reference Raab, Hartley, Morris, Bornstein and Shenhav2018), but understanding why these shifts occur has proven more difficult.
Disentangling whether age-related changes in learning reflect adaptation to age-varying “external” ecological problems or “internal” changes in cognitive capacity has been difficult, in part because developmentalists have lacked the formalizations needed to make specific predictions about how these two factors might drive age-related changes in behavior. As an example, in many studies of reinforcement learning, participants are tasked with earning the most reward (e.g., points or money) by selecting between different options. In these tasks, children tend to make “noisier” or more exploratory choices, which typically result in poorer performance (Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019) but enhanced learning of task structure (Blanco & Sloutsky, Reference Blanco and Sloutsky2021; Liquin & Gopnik, Reference Liquin and Gopnik2022; Sumner et al., Reference Sumner, Li, Perfors, Hayes, Navarro and Sarnecka2019). These findings have led researchers to theorize that children and adults have optimized their learning computations to maximize reward over different types of environments – children's learning contexts may have features (e.g., greater reward stochasticity, longer temporal horizons) that favor exploration over immediate reward gain (Gopnik, Reference Gopnik2020). An alternative account, however, is that children's “noisier” behavior reflects a more limited cognitive capacity (Craik & Bialystok, Reference Craik and Bialystok2006; Ruel, Devine, & Eppinger, Reference Ruel, Devine and Eppinger2021), which constrains their “optimization” of learning computations.
By enabling separable manipulation of external experience and internal cognitive capacity, meta-learned models may help arbitrate between these accounts of developmental change. As Binz et al. note, through meta-learning, the optimal algorithm emerges through experience, and is shaped by the distribution of environments on which the model is trained. Thus, these models can be used to test how differences in “training” experience might yield different patterns of learning at “test.” Examining how training environments influence model behavior within the same tasks that have been used to study cognitive development can provide insight into the types of environments for which children, adolescents, and adults have optimized their learning computations. As one example, Binz and Schulz (Reference Binz and Schulz2022) found that increasing the variance in the rewards that a model experienced during training led to exploration decisions at test that more closely approximated those of human learners, suggesting that people may have tuned their learning computations for environments with more stochasticity than the task context. Importantly, meta-learned models can provide insight into how optimal learning is shaped by exposure to highly complex environments in which multiple features vary and interact – environments for which analytically derived “solutions” are intractable but that are more reflective of real-world contexts than the simple tasks that have been used in most prior research.
At the same time, meta-learned models can account for how changes in capacity limitations may yield developmental changes in learning. As Binz et al. note, the complexity of meta-learned models can be manipulated easily, particularly when they are implemented as neural networks. Manipulating capacity constraints and comparing model behavior to that of learners at different ages can thus provide insight into whether developmental changes in learning are well accounted for by theories of “resource-rationality.” Binz and Schulz (Reference Binz and Schulz2022) found, for example, that increasing the complexity of the algorithms that a meta-learning network could implement led to changes in patterns of directed exploration that mirrored those that occur across adolescence (Somerville et al., Reference Somerville, Sasse, Garrad, Drysdale, Abi Akar, Insel and Wilson2017). This result exemplifies how, rather than emerging from differences in external “training” experience, age-related changes in learning can also be driven by differences in cognitive capacity.
Further, neural network implementations of meta-learning enable separable manipulations of “algorithmic complexity” via network weights and “computational complexity” via network activations. Future developmental modeling work could explore the ramifications of constraints on algorithmic versus computational complexity. Numerous studies have revealed a dissociation between children's knowledge of rules or structure and their ability to leverage them to guide behavior (Decker, Otto, Daw, & Hartley, Reference Decker, Otto, Daw and Hartley2016; Zelazo, Frye, & Rapus, Reference Zelazo, Frye and Rapus1996). The types of structural knowledge that can be acquired at different ages may depend on algorithmic complexity, while use of that knowledge may rely on processes like working memory and proactive cognitive control, which may instantiated via complex computations. Behavioral dissociations between algorithmic and computational complexity may be mirrored by neural dissociations. Age-related change in brain structure or the wiring of neural circuitry may be analogous to age-related change in network weights and relate more strongly to algorithmic complexity, whereas age-related change in patterns of neural activation during learning may be more closely related to the network activations that underlie computational complexity. Thus, predictions about how age-related change in these two forms of capacity limitations affect learning could be further tested and constrained with neural data.
While we have suggested that researchers can leverage meta-learned models to more explicitly test whether children optimize their behavior for different environments or with different constraints, this dichotomy is likely false. Experience and constraints interact throughout the lifespan, and the changing “constraints” implemented by neurobiology may themselves serve an adaptive function – it may be the case that the complexity of the learning algorithms an organism can implement and execute systematically increases through exposure to increasingly varied and complex environments. Meta-learned models of cognition thus have the potential to address questions of longstanding interest in developmental science, while empirical developmental research provides a valuable testbed for the theoretical utility of these computational tools.
You have
Access
Binz et al. argue compellingly for meta-learning as a tool to understand adult cognition, but their vision is incomplete: Meta-learned models are particularly apt tools for studying development. As the authors note, meta-learned models provide a natural foundation for theorizing about learning to learn (Wang, Reference Wang2021) and for understanding experience-driven changes in goal-directed behavior (Nussenbaum & Hartley, Reference Nussenbaum and Hartleyin press). Here, however, we consider the authors’ central argument and focus not on meta-learning algorithms or the process of “learning to learn,” but rather on meta-learned models as a tool for understanding developmental changes in learning. To date, research has revealed age-related shifts in the algorithms and neural circuitry that underlie learning (Bolenz, Reiter, & Eppinger, Reference Bolenz, Reiter and Eppinger2017; Gualtieri & Finn, Reference Gualtieri and Finn2022; Hartley, Nussenbaum, & Cohen, Reference Hartley, Nussenbaum and Cohen2021; Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019; Raab & Hartley, Reference Raab, Hartley, Morris, Bornstein and Shenhav2018), but understanding why these shifts occur has proven more difficult.
Disentangling whether age-related changes in learning reflect adaptation to age-varying “external” ecological problems or “internal” changes in cognitive capacity has been difficult, in part because developmentalists have lacked the formalizations needed to make specific predictions about how these two factors might drive age-related changes in behavior. As an example, in many studies of reinforcement learning, participants are tasked with earning the most reward (e.g., points or money) by selecting between different options. In these tasks, children tend to make “noisier” or more exploratory choices, which typically result in poorer performance (Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019) but enhanced learning of task structure (Blanco & Sloutsky, Reference Blanco and Sloutsky2021; Liquin & Gopnik, Reference Liquin and Gopnik2022; Sumner et al., Reference Sumner, Li, Perfors, Hayes, Navarro and Sarnecka2019). These findings have led researchers to theorize that children and adults have optimized their learning computations to maximize reward over different types of environments – children's learning contexts may have features (e.g., greater reward stochasticity, longer temporal horizons) that favor exploration over immediate reward gain (Gopnik, Reference Gopnik2020). An alternative account, however, is that children's “noisier” behavior reflects a more limited cognitive capacity (Craik & Bialystok, Reference Craik and Bialystok2006; Ruel, Devine, & Eppinger, Reference Ruel, Devine and Eppinger2021), which constrains their “optimization” of learning computations.
By enabling separable manipulation of external experience and internal cognitive capacity, meta-learned models may help arbitrate between these accounts of developmental change. As Binz et al. note, through meta-learning, the optimal algorithm emerges through experience, and is shaped by the distribution of environments on which the model is trained. Thus, these models can be used to test how differences in “training” experience might yield different patterns of learning at “test.” Examining how training environments influence model behavior within the same tasks that have been used to study cognitive development can provide insight into the types of environments for which children, adolescents, and adults have optimized their learning computations. As one example, Binz and Schulz (Reference Binz and Schulz2022) found that increasing the variance in the rewards that a model experienced during training led to exploration decisions at test that more closely approximated those of human learners, suggesting that people may have tuned their learning computations for environments with more stochasticity than the task context. Importantly, meta-learned models can provide insight into how optimal learning is shaped by exposure to highly complex environments in which multiple features vary and interact – environments for which analytically derived “solutions” are intractable but that are more reflective of real-world contexts than the simple tasks that have been used in most prior research.
At the same time, meta-learned models can account for how changes in capacity limitations may yield developmental changes in learning. As Binz et al. note, the complexity of meta-learned models can be manipulated easily, particularly when they are implemented as neural networks. Manipulating capacity constraints and comparing model behavior to that of learners at different ages can thus provide insight into whether developmental changes in learning are well accounted for by theories of “resource-rationality.” Binz and Schulz (Reference Binz and Schulz2022) found, for example, that increasing the complexity of the algorithms that a meta-learning network could implement led to changes in patterns of directed exploration that mirrored those that occur across adolescence (Somerville et al., Reference Somerville, Sasse, Garrad, Drysdale, Abi Akar, Insel and Wilson2017). This result exemplifies how, rather than emerging from differences in external “training” experience, age-related changes in learning can also be driven by differences in cognitive capacity.
Further, neural network implementations of meta-learning enable separable manipulations of “algorithmic complexity” via network weights and “computational complexity” via network activations. Future developmental modeling work could explore the ramifications of constraints on algorithmic versus computational complexity. Numerous studies have revealed a dissociation between children's knowledge of rules or structure and their ability to leverage them to guide behavior (Decker, Otto, Daw, & Hartley, Reference Decker, Otto, Daw and Hartley2016; Zelazo, Frye, & Rapus, Reference Zelazo, Frye and Rapus1996). The types of structural knowledge that can be acquired at different ages may depend on algorithmic complexity, while use of that knowledge may rely on processes like working memory and proactive cognitive control, which may instantiated via complex computations. Behavioral dissociations between algorithmic and computational complexity may be mirrored by neural dissociations. Age-related change in brain structure or the wiring of neural circuitry may be analogous to age-related change in network weights and relate more strongly to algorithmic complexity, whereas age-related change in patterns of neural activation during learning may be more closely related to the network activations that underlie computational complexity. Thus, predictions about how age-related change in these two forms of capacity limitations affect learning could be further tested and constrained with neural data.
While we have suggested that researchers can leverage meta-learned models to more explicitly test whether children optimize their behavior for different environments or with different constraints, this dichotomy is likely false. Experience and constraints interact throughout the lifespan, and the changing “constraints” implemented by neurobiology may themselves serve an adaptive function – it may be the case that the complexity of the learning algorithms an organism can implement and execute systematically increases through exposure to increasingly varied and complex environments. Meta-learned models of cognition thus have the potential to address questions of longstanding interest in developmental science, while empirical developmental research provides a valuable testbed for the theoretical utility of these computational tools.
Acknowledgements
We thank Rheza Budiono for helpful feedback.
Financial support
Preparation of this commentary was supported by a CV Starr Foundation Fellowship (to K. N.).
Competing interest
None.