Book contents
- The Drive for Knowledge
- The Drive for Knowledge
- Copyright page
- Contents
- Figures
- Tables
- Notes on Contributors
- Preface
- Part I What Drives Humans to Seek Information?
- Part II How Do Humans Search for Information?
- Chapter 5 What Makes a Good Query?
- Chapter 6 Active Inference, Bayesian Optimal Design, and Expected Utility
- Chapter 7 Exploration Beyond Bandits
- Chapter 8 Representational Exchange in Human Social Learning
- Part III Which Machinery Supports the Drive for Knowledge?
- Index
- References
Chapter 8 - Representational Exchange in Human Social Learning
Balancing Efficiency and Flexibility
from Part II - How Do Humans Search for Information?
Published online by Cambridge University Press: 19 May 2022
Book contents
- The Drive for Knowledge
- The Drive for Knowledge
- Copyright page
- Contents
- Figures
- Tables
- Notes on Contributors
- Preface
- Part I What Drives Humans to Seek Information?
- Part II How Do Humans Search for Information?
- Chapter 5 What Makes a Good Query?
- Chapter 6 Active Inference, Bayesian Optimal Design, and Expected Utility
- Chapter 7 Exploration Beyond Bandits
- Chapter 8 Representational Exchange in Human Social Learning
- Part III Which Machinery Supports the Drive for Knowledge?
- Index
- References
Summary
What makes human social learning so powerful? While past accounts have sometimes prioritized finding the single capacity that makes the largest difference, our social learning abilities span a wide spectrum of capacities from the high-fidelity imitation of behaviors to inferring and learning from hidden mental states. Here, we propose that the power of human social learning lies not within a single capacity, but in our ability to flexibly arbitrate between different computations and to integrate their outputs. In particular, learners can directly copy the demonstrator’s actions in the absence of causal insight (policy imitation), infer their instrumental values (value inference), or infer their model of the world and intrinsic rewards (belief inference and reward inference). Each of these strategies trades off the cost of computation against the flexibility and compositionality of its outputs. Crucially, we have the capacity to arbitrate and exchange information between these representational formats. Human social learning, we suggest, is powerful not just because of the way it moves information between minds, but also because of the way it flexibly moves information within them.
- Type
- Chapter
- Information
- The Drive for KnowledgeThe Science of Human Information Seeking, pp. 169 - 192Publisher: Cambridge University PressPrint publication year: 2022
References
Apperly, I. (2010). Mindreaders: The cognitive basis of “theory of mind.” Psychology Press.CrossRefGoogle Scholar
Atkisson, C., O’Brien, M. J., & Mesoudi, A. (2012). Adult learners in a novel environment use prestige-biased social learning. Evolutionary Psychology: An International Journal of Evolutionary Approaches to Psychology and Behavior, 10(3), 519–537.CrossRefGoogle Scholar
Baker, C. L., Jara-Ettinger, J., Saxe, R., & Tenenbaum, J. B. (2017). Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4), 1–10.CrossRefGoogle Scholar
Bandura, A. (1962). Social learning through imitation. Nebraska Symposium on Motivation, 330, 211–274.Google Scholar
Bhui, R., Lai, L., & Gershman, S. J. (2021). Resource-rational decision making. Current Opinion in Behavioral Sciences, 41, 15–21.Google Scholar
Botvinick, M., & Weinstein, A. (2014). Model-based hierarchical reinforcement learning and human action control. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369(1655). https://doi.org/10.1098/rstb.2013.0480.Google Scholar
Boyd, R., & Richerson, P. J. (1988). Culture and the Evolutionary Process. University of Chicago Press.Google Scholar
Catmur, C., Walsh, V., & Heyes, C. (2009). Associative sequence learning: The role of experience in the development of imitation and the mirror system. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1528), 2369–2380.Google Scholar
Charpentier, C. J., Iigaya, K., & O’Doherty, J. P. (2020). A neuro-computational account of arbitration between choice imitation and goal emulation during human observational learning. Neuron, 106(4), 687–699.e7.Google Scholar
Cogliati Dezza, I., Cleeremans, A., & Alexander, W. (2019). Should we control? The interplay between cognitive control and information integration in the resolution of the exploration-exploitation dilemma. Journal of Experimental Psychology. General, 148(6), 977–993.Google Scholar
Collette, S., Pauli, W. M., Bossaerts, P., & O’Doherty, J. (2017). Neural computations underlying inverse reinforcement learning in the human brain. eLife, 6. https://doi.org/10.7554/eLife.29718.Google Scholar
Cushman, F. (2020). Rationalization is rational. The Behavioral and Brain Sciences, 43, e28.Google Scholar
Cushman, F., & Morris, A. (2015). Habitual control of goal selection in humans. Proceedings of the National Academy of Sciences of the United States of America, 112(45), 13817–13822.Google Scholar
Dasgupta, I., & Gershman, S. J. (2021). Memory as a computational resource. Trends in Cognitive Sciences, 25(3), 240–251.Google Scholar
Dasgupta, I., Schulz, E., Goodman, N. D., & Gershman, S. J. (2018). Remembrance of inferences past: Amortization in human hypothesis generation. Cognition, 178, 67–81.Google Scholar
Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.Google Scholar
Derex, M., Bonnefon, J.-F., Boyd, R., & Mesoudi, A. (2019). Causal understanding is not necessary for the improvement of culturally evolving technology. Nature Human Behaviour, 3(5), 446–452.CrossRefGoogle Scholar
Dezfouli, A., & Balleine, B. W. (2013). Actions, action sequences and habits: Evidence that goal-directed and habitual action control are hierarchically organized. PLoS Computational Biology, 9(12), e1003364.Google Scholar
Foster, D. J. (2017). Replay comes of age. Annual Review of Neuroscience, 40, 581–602.Google Scholar
Gergely, G., & Csibra, G. (2003). Teleological reasoning in infancy: The naıve theory of rational action. In Trends in Cognitive Sciences (Vol. 7, Issue 7, pp. 287–292). https://doi.org/10.1016/s1364-6613(03)00128-1.Google Scholar
Gershman, S. J. (2020). Origin of perseveration in the trade-off between reward and complexity. Cognition, 204, 104394.Google Scholar
Gershman, S. J., Horvitz, E. J., & Tenenbaum, J. B. (2015). Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 349(6245), 273–278.Google Scholar
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014). Retrospective revaluation in sequential decision making: A tale of two systems. Journal of Experimental Psychology. General, 143(1), 182–194.Google Scholar
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482.Google Scholar
Gweon, H. (2021). Inferential Social Learning: How humans learn from others and help others learn. https://doi.org/10.31234/osf.io/8n34t.Google Scholar
Hayden, B. Y., & Niv, Y. (2021). The case against economic values in the orbitofrontal cortex (or anywhere else in the brain). Behavioral Neuroscience, 135(2), 192–201.Google Scholar
Henrich, J. (2017). The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter. Princeton University Press.Google Scholar
Henrich, J., & Gil-White, F. J. (2001). The evolution of prestige: Freely conferred deference as a mechanism for enhancing the benefits of cultural transmission. Evolution and Human Behavior: Official Journal of the Human Behavior and Evolution Society, 22(3), 165–196.CrossRefGoogle ScholarPubMed
Herrmann, E., Call, J., Hernàndez-Lloreda, M. V., Hare, B., & Tomasello, M. (2007). Humans have evolved specialized skills of social cognition: the cultural intelligence hypothesis. Science, 317(5843), 1360–1366.Google Scholar
Heyes, C. (2001). Causes and consequences of imitation. Trends in Cognitive Sciences, 5(6), 253–261.CrossRefGoogle ScholarPubMed
Heyes, C. (2002). Transformational and associative theories of imitation. Imitation in Animals and Artifacts, 607, 501–523.Google Scholar
Heyes, C. (2018). Cognitive Gadgets: The Cultural Evolution of Thinking. Harvard University Press.Google Scholar
Ho, M. K., MacGlashan, J., Littman, M. L., & Cushman, F. (2017). Social is special: A normative framework for teaching with and learning from evaluative feedback. Cognition, 167, 91–106.CrossRefGoogle ScholarPubMed
Hoppitt, W., & Laland, K. N. (2013). Social Learning: An Introduction to Mechanisms, Methods, and Models. Princeton University Press.Google Scholar
Horner, V., & Whiten, A. (2005). Causal knowledge and imitation/emulation switching in chimpanzees (Pan troglodytes) and children (Homo sapiens). Animal Cognition, 8(3), 164–181.Google Scholar
Huys, Q. J. M., Lally, N., Faulkner, P., Eshel, N., Seifritz, E., Gershman, S. J., Dayan, P., & Roiser, J. P. (2015). Interplay of approximate planning strategies. Proceedings of the National Academy of Sciences of the United States of America, 112(10), 3098–3103.Google Scholar
Jara-Ettinger, J. (2019). Theory of mind as inverse reinforcement learning. Current Opinion in Behavioral Sciences, 29, 105–110.Google Scholar
Jara-Ettinger, J., Gweon, H., Schulz, L. E., & Tenenbaum, J. B. (2016). The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology. Trends in Cognitive Sciences, 20(8), 589–604.CrossRefGoogle ScholarPubMed
Jara-Ettinger, J., Gweon, H., Tenenbaum, J. B., & Schulz, L. E. (2015). Children’s understanding of the costs and rewards underlying rational action. Cognition, 140, 14–23.CrossRefGoogle ScholarPubMed
Jern, A., Lucas, C. G., & Kemp, C. (2017). People learn other people’s preferences through inverse decision-making. Cognition, 168, 46–64.Google Scholar
Jiménez, Á. V., & Mesoudi, A. (2019). Prestige-biased social learning: Current evidence and outstanding questions. Palgrave Communications, 5(1), 20.Google Scholar
Keramati, M., Smittenaar, P., Dolan, R. J., & Dayan, P. (2016). Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum. Proceedings of the National Academy of Sciences of the United States of America, 113(45), 12868–12873.Google Scholar
Kool, W., Cushman, F. A., & Gershman, S. J. (2018). Competition and cooperation between multiple reinforcement learning systems. In Morris, R., Bornstein, A., & Shenhav, A. (Eds.), Goal-directed decision making (pp. 153–178). Academic Press.Google Scholar
Kool, W., Gershman, S. J., & Cushman, F. A. (2017). Cost-benefit arbitration between multiple reinforcement-learning systems. Psychological Science, 28(9), 1321–1333.Google Scholar
Kool, W., Gershman, S. J., & Cushman, F. A. (2018). Planning complexity registers as a cost in metacontrol. Journal of Cognitive Neuroscience, 30(10), 1391–1404.Google Scholar
Legare, C. H., & Nielsen, M. (2015). Imitation and innovation: The dual engines of cultural learning. Trends in Cognitive Sciences, 19(11), 688–699.Google Scholar
Lieder, F., & Griffiths, T. L. (2020). Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. In Behavioral and Brain Sciences (Vol. 43). https://doi.org/10.1017/s0140525x1900061x.Google Scholar
Liu, S., Brooks, N. B., & Spelke, E. S. (2019). Origins of the concepts cause, cost, and goal in prereaching infants. Proceedings of the National Academy of Sciences of the United States of America, 116(36), 17747–17752.Google Scholar
Lyons, D. E., Young, A. G., & Keil, F. C. (2007). The hidden structure of overimitation. Proceedings of the National Academy of Sciences of the United States of America, 104(50), 19751–19756.Google Scholar
Maisto, D., Friston, K., & Pezzulo, G. (2019). Caching mechanisms for habit formation in active inference. Neurocomputing, 359, 298–314.Google Scholar
McGuigan, N., Whiten, A., Flynn, E., & Horner, V. (2007). Imitation of causally opaque versus causally transparent tool use by 3- and 5-year-old children. Cognitive Development, 22(3), 353–364.Google Scholar
Miller, K. J., Botvinick, M. M., & Brody, C. D. (2017). Dorsal hippocampus contributes to model-based planning. Nature Neuroscience. https://doi.org/10.1101/096594.Google Scholar
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019). Habits without values. Psychological Review, 126(2), 292–311.Google Scholar
Miller, N. E., & Dollard, J. (1941). Social Learning and Imitation (Vol. 55). Yale University Press.Google Scholar
Momennejad, I., Otto, A. R., Daw, N. D., & Norman, K. A. (2018). Offline replay supports planning in human reinforcement learning. eLife, 7. https://doi.org/10.7554/eLife.32548.CrossRefGoogle ScholarPubMed
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M. M., Daw, N. D., & Gershman, S. J. (2017). The successor representation in human reinforcement learning. Nature Human Behaviour, 1(9), 680–692.CrossRefGoogle ScholarPubMed
Morris, A., & Cushman, F. (2018). A common framework for theories of norm compliance. Social Philosophy & Policy, 35(1), 101–127.Google Scholar
Najar, A., Bonnet, E., Bahrami, B., & Palminteri, S. (2020). The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning. PLoS Biology, 18(12), e3001028.Google Scholar
O’Donnell, T. J. (2015). Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage. MIT Press.Google Scholar
Otto, A. R., Gershman, S. J., Markman, A. B., & Daw, N. D. (2013). The curse of planning: Dissecting multiple reinforcement-learning systems by taxing the central executive. Psychological Science, 24(5), 751–761.Google Scholar
Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013). Working-memory capacity protects model-based learning from stress. Proceedings of the National Academy of Sciences of the United States of America, 110(52), 20941–20946.Google Scholar
Rendell, L., Boyd, R., Cownden, D., Enquist, M., Eriksson, K., Feldman, M. W., … & Laland, K. N. (2010). Why copy others? Insights from the social learning strategies tournament. Science, 328(5975), 208–213.Google Scholar
Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive Science, 26(5), 521–562.Google Scholar
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017). Predictive representations can link model-based reinforcement learning to model-free mechanisms. PLoS Computational Biology, 13(9), e1005768.Google Scholar
Scott-Phillips, T. C. (2017). A (simple) experimental demonstration that cultural evolution is not replicative, but reconstructive – and an explanation of why this difference matters. Journal of Cognition and Culture, 17(1–2), 1–11.Google Scholar
Shafto, P., Goodman, N. D., & Frank, M. C. (2012). Learning from others: The consequences of psychological reasoning for human learning. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(4), 341–351.Google Scholar
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013). The expected value of control: An integrative theory of anterior cingulate cortex function. Neuron, 79(2), 217–240.Google Scholar
Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57(4), 193–216.Google Scholar
Solway, A., & Botvinick, M. M. (2012). Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates. Psychological Review, 119(1), 120–154.Google Scholar
Solway, A., & Botvinick, M. M. (2015). Evidence integration in model-based tree search. Proceedings of the National Academy of Sciences of the United States of America, 112(37), 11708–11713.Google Scholar
Solway, A., Diuk, C., Córdova, N., Yee, D., Barto, A. G., Niv, Y., & Botvinick, M. M. (2014). Optimal behavioral hierarchy. PLoS Computational Biology, 10(8), e1003779.Google Scholar
Sperber, D. (2006). Why a deep understanding of cultural evolution is incompatible with shallow psychology. In Enfield, N. J. & Levinson, Stephen C. (Ed.), Roots of human sociality (pp. 431–449). Routledge.Google Scholar
Strachan, J., Curioni, A., Constable, M., Knoblich, G., & Charbonneau, M. (2020). A methodology for distinguishing copying and reconstruction in cultural transmission episodes. In Denison, S, Mack, M, Xu, Y, Yang, A and Armstrong, C. B (Eds.), Proceedings of the 42nd Annual Conference of the Cognitive Science Society. https://researchportal.northumbria.ac.uk/ws/files/32896647/0831.pdf.Google Scholar
Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning, second edition: An Introduction. MIT Press.Google Scholar
Tennie, C., Call, J., & Tomasello, M. (2009). Ratcheting up the ratchet: On the evolution of cumulative culture. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1528), 2405–2415.Google Scholar
Thorndike, E. L. (1932). The fundamentals of learning. https://psycnet.apa.org/record/2006-04535-000.Google Scholar
Tolman, E. C. (1948). Cognitive maps in rats and men. Psychological Review (Vol. 55, Issue 4, pp. 189–208). https://doi.org/10.1037/h0061626.Google Scholar
Tomasello, M. (1996). Do apes ape. In Heyes, C. M & Galef, B. G, Jr. (Eds.), Social Learning in Animals: The Roots of Culture, (pp. 319–346). Academic Press. https://doi.org/10.1016/B978-012273965-1/50016-9.Google Scholar
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. The Behavioral and Brain Sciences, 28(5), 675–691; discussion 691–735.Google Scholar
Tomasello, M., Davis-Dasilva, M., Camak, L., & Bard, K. (1987). Observational learning of tool-use by young chimpanzees. Human Evolution, 2(2), 175–183.Google Scholar
Vélez, N., & Gweon, H. (2021). Learning from other minds: An optimistic critique of reinforcement learning models of social learning. Current Opinion in Behavioral Sciences, 38, 110–115.Google Scholar
Vikbladh, O. M., Meager, M. R., King, J., Blackmon, K., Devinsky, O., Shohamy, D., Burgess, N., & Daw, N. D. (2019). Hippocampal contributions to model-based planning and spatial memory. Neuron, 102(3), 683–693.e4.Google Scholar
Whiten, A., & Ham, R. (1992). Kingdom: Reappraisal of a century of research. Advances in the Study of Behavior, 21, 239.Google Scholar
Wu, C. M., Schulz, E., Gerbaulet, K., Pleskac, T. J., & Speekenbrink, M. (2021). Time to explore: Adaptation of exploration under time pressure. PsyArXiv. https://doi.org/10.31234/osf.io/dsw7q.Google Scholar
Zaki, J., Schirmer, J., & Mitchell, J. P. (2011). Social influence modulates the neural computation of value. Psychological Science, 22(7), 894–900.Google Scholar
- 7
- Cited by