Hostname: page-component-f554764f5-68cz6 Total loading time: 0 Render date: 2025-04-22T08:38:52.648Z Has data issue: false hasContentIssue false
  • English
  • Français

The reinforcement metalearner as a biologically plausible meta-learning framework

Published online by Cambridge University Press:  23 September 2024

Tim Vriens Open the ORCID record for Tim Vriens [Opens in a new window] ,
Mattias Horan Open the ORCID record for Mattias Horan [Opens in a new window] ,
Jacqueline Gottlieb Open the ORCID record for Jacqueline Gottlieb [Opens in a new window]  and
Massimo Silvetti
Tim Vriens
Affiliation:
Institute of Cognitive Sciences and Technologies, CNR, Rome, Italy [email protected], [email protected] https://ctnlab.it/index.php/massimo-silvetti/, https://www.istc.cnr.it/en/people/massimo-silvetti
Mattias Horan
Affiliation:
Sainsbury Wellcome Centre, University College London, London, UK [email protected],
Jacqueline Gottlieb*
Affiliation:
Department of Neuroscience, Columbia University, New York, NY, USA [email protected], https://zuckermaninstitute.columbia.edu/jacqueline-gottlieb-phd Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
Massimo Silvetti
Affiliation:
Institute of Cognitive Sciences and Technologies, CNR, Rome, Italy [email protected], [email protected] https://ctnlab.it/index.php/massimo-silvetti/, https://www.istc.cnr.it/en/people/massimo-silvetti
*
*Corresponding author.
Get access Rights & Permissions [Opens in a new window]

Abstract

We argue that the type of meta-learning proposed by Binz et al. generates models with low interpretability and falsifiability that have limited usefulness for neuroscience research. An alternative approach to meta-learning based on hyperparameter optimization obviates these concerns and can generate empirically testable hypotheses of biological computations.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 47 , 2024 , e168
Check for updates
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Castelvecchi, D. (2016). Can we open the black box of AI? Nature, 538, 20–23. https://doi.org/10.1038/538020aCrossRefGoogle ScholarPubMed
Daddaoua, N., Lopes, M., & Gottlieb, J. (2016). Intrinsically motivated oculomotor exploration guided by uncertainty reduction and conditioned reinforcement in non-human primates. Scientific Reports, 6(1), Article 1. https://doi.org/10.1038/srep20202CrossRefGoogle ScholarPubMed
Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4–6), 495–506. https://doi.org/10.1016/s0893-6080(02)00044-8CrossRefGoogle ScholarPubMed
Horan, M., Daddaoua, N., & Gottlieb, J. (2019). Parietal neurons encode information sampling based on decision uncertainty. Nature Neuroscience, 22(8), 1327–1335. https://doi.org/10.1038/s41593-019-0440-1CrossRefGoogle ScholarPubMed
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366. https://doi.org/10.1016/0893-6080(89)90020-8CrossRefGoogle Scholar
Marblestone, A. H., Wayne, G., & Kording, K. P. (2017). Understand the cogs to understand cognition. Behavioral and Brain Sciences, 40, e272. https://doi.org/10.1017/S0140525X17000218CrossRefGoogle ScholarPubMed
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain, and cognitive control in the cingulate cortex. Nature Reviews. Neuroscience, 12(3), 154–167. https://doi.org/10.1038/nrn2994CrossRefGoogle ScholarPubMed
Silvetti, M., Lasaponara, S., Daddaoua, N., Horan, M., & Gottlieb, J. (2023). A reinforcement meta-learning framework of executive function and information demand. Neural Networks, 157, 103–113. https://doi.org/10.1016/j.neunet.2022.10.004CrossRefGoogle ScholarPubMed
Silvetti, M., Seurinck, R., van Bochove, M., & Verguts, T. (2013). The influence of the noradrenergic system on optimal control of neural plasticity. Frontiers in Behavioral Neuroscience, 7, 160. https://www.frontiersin.org/articles/10.3389/fnbeh.2013.00160CrossRefGoogle ScholarPubMed
Silvetti, M., Vassena, E., Abrahamse, E., & Verguts, T. (2018). Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner. PLoS Computational Biology, 14(8), e1006370. https://doi.org/10.1371/journal.pcbi.1006370CrossRefGoogle ScholarPubMed
Varazzani, C., San-Galli, A., Gilardeau, S., & Bouret, S. (2015). Noradrenaline and dopamine neurons in the reward/effort trade-off: A direct electrophysiological comparison in behaving monkeys. The Journal of Neuroscience, 35(20), 7866–7877. https://doi.org/10.1523/JNEUROSCI.0454-15.2015CrossRefGoogle ScholarPubMed
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8), 665–670. https://doi.org/10.1038/nmeth.1635CrossRefGoogle ScholarPubMed