Hostname: page-component-745bb68f8f-cphqk Total loading time: 0 Render date: 2025-02-11T18:04:56.825Z Has data issue: false hasContentIssue false
  • English
  • Français

An empirical generative framework for computational modeling of language acquisition*

Published online by Cambridge University Press:  26 April 2010

Heidi R. Waterfall ,
Ben Sandbank ,
Luca Onnis  and
Shimon Edelman
Heidi R. Waterfall
Affiliation:
Department of Psychology, Cornell University and Department of Psychology, University of Chicago
Ben Sandbank
Affiliation:
School of Computer Science, Tel Aviv University
Luca Onnis
Affiliation:
Department of Second Language Studies, University of Hawaii
Shimon Edelman
Affiliation:
Department of Psychology, Cornell University and Department of Brain and Cognitive Engineering, Korea University
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper reports progress in developing a computer model of language acquisition in the form of (1) a generative grammar that is (2) algorithmically learnable from realistic corpus data, (3) viable in its large-scale quantitative performance and (4) psychologically real. First, we describe new algorithmic methods for unsupervised learning of generative grammars from raw CHILDES data and give an account of the generative performance of the acquired grammars. Next, we summarize findings from recent longitudinal and experimental work that suggests how certain statistically prominent structural properties of child-directed speech may facilitate language acquisition. We then present a series of new analyses of CHILDES data indicating that the desired properties are indeed present in realistic child-directed speech corpora. Finally, we suggest how our computational results, behavioral findings, and corpus-based insights can be integrated into a next-generation model aimed at meeting the four requirements of our modeling framework.

Type
Articles
Information
Journal of Child Language , Volume 37 , Special Issue 3: Computational models of child language learning , June 2010 , pp. 671 - 703
Copyright
Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adriaans, P. (2001). Learning shallow context free languages under simple distributions. In Copestake, A. & Vermeulen, K. (eds), Algebras, diagrams and decisions in language, logic and computation, 1–35/ Stanford, CA: CSLI/CUP.Google Scholar
Baayen, R. (2006). Analyzing linguistic data: A practical introduction to statistics using R. Cambridge: Cambridge University Press.Google Scholar
Baker, L. & McCallum, A. (1998). Distributional clustering of words for text classification. In SIGIR-98, 21st ACM International Conference on Research and Development in Information Retrieval, Melbourne, 96–103. New York: ACM Press.Google Scholar
Batchelder, E. (2002). Bootstrapping the lexicon: A computational model of infant speech segmentation. Cognition 83, 167–206.CrossRefGoogle ScholarPubMed
Bloom, L. (1970). Language development: Form and function in emerging grammars. Cambridge, MA: MIT Press.Google Scholar
Bloom, L. (1973). One word at a time: The use of single-word utterances before syntax. The Hague: Mouton.Google Scholar
Bod, R. (2009). Constructions at work or at rest. Cognitive Linguistics 20, 129–34.CrossRefGoogle Scholar
Brodsky, P., Waterfall, H. & Edelman, S. (2007). Characterizing motherese: On the computational structure of child directed language. In McNamara, D. & Trafton, J. (eds), Proceedings of the 29th Cognitive Science Society Conference, Nashville, 833–38. Austin, TX: Cognitive Science Society.Google Scholar
Brown, R. (1973). A first language: The early stages. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Brown, R., Cazden, C. & Bellugi, U. (1969). The child's grammar from one to three. In Hill, J. P. (ed.), Minnesota symposium on child development, Volume 2, 28–73. Minneapolis, MI: University of Minnesota Press.Google Scholar
Charniak, E. (1997). Statistical techniques for natural language parsing. AI Magazine 18, 33–44.Google Scholar
Chater, N. & Vitányi, P. (2007). Ideal learning of natural language: Positive results about learning from positive evidence. Journal of Mathematical Psychology 51, 135–63.CrossRefGoogle Scholar
Chomsky, N. (1957). Syntactic structures. The Hague: Mouton.CrossRefGoogle Scholar
Chomsky, N. (1995). The Minimalist Program. Cambridge, MA: MIT Press.Google Scholar
Christiansen, M. & Chater, N. (2001). Connectionist psycholinguistics: Capturing the empirical data. Trends in Cognitive Sciences 5, 82–88.CrossRefGoogle ScholarPubMed
Clark, A. (2001). Unsupervised language acquisition: Theory and practice. Unpublished PhD thesis, School of Cognitive and Computing Sciences, University of Sussex.Google Scholar
Clark, A. (2006). PAC learning unambiguous NTS languages. In Proceedings of ICGI, 59–71. Tokyo: Springer-Verlag.Google Scholar
Demuth, K., Culbertson, J. & Alter, J. (2006). Word-minimality, epenthesis, and coda licensing in the acquisition of English. Language & Speech 49, 137–74.CrossRefGoogle ScholarPubMed
Edelman, S. (2010). On look-ahead in language: Navigating a multitude of familiar paths. In Bar, M. (ed.), Prediction in the brain (to appear). New York: Oxford University Press.Google Scholar
Edelman, S. & Waterfall, H. (2007). Behavioral and computational aspects of language and its acquisition. Physics of Life Reviews 4, 253–77.CrossRefGoogle Scholar
Elman, J., Bates, E., Johnson, M., Karmiloff-Smith, A., Parisi, D. & Plunkett, K. (1996). Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press.Google Scholar
Goldsmith, J. (2007). Towards a new empiricism. In de Carvalho, J. B. (ed.), Recherches linguistiques à Vincennes, Volume 36.Google Scholar
Goodman, J. (2001). A bit of progress in language modeling. Computer Speech and Language 15, 403–434.CrossRefGoogle Scholar
Hall, W., Nagy, W. & Linn, R. (1984). Spoken words: Effects of situation and social group on oral word usage and frequency. Hillsdale, NJ: Erlbaum.Google Scholar
Hall, W., Nagy, W. & Nottenburg, G. (1981). Situational variation in the use of internal state words. Champaign, IL: University of Illinois.Google Scholar
Hall, W. & Tirre, W. (1979). The communicative environment of young children: Social class, ethnic and situational differences. Champaign, IL: University of Illinois.Google Scholar
Harris, Z. (1954). Distributional structure. Word 10, 140–62.CrossRefGoogle Scholar
Higginson, R. (1985). Fixing-assimilation in language acquisition. Unpublished doctoral dissertation, Washington State University.Google Scholar
Hoff-Ginsberg, E. (1985). Some contributions of mothers' speech to their children's syntactic growth. Journal of Child Language 12, 367–85.CrossRefGoogle ScholarPubMed
Hoff-Ginsberg, E. (1990). Maternal speech and the child's development of syntax: A further look. Journal of Child Language 17, 85–99.CrossRefGoogle ScholarPubMed
Joshi, A. & Schabes, Y. (1997). Tree-adjoining grammars. In Rozenberg, G. and Salomaa, A. (eds), Handbook of formal languages, 3, 69–124. Berlin: Springer.CrossRefGoogle Scholar
Klein, D. & Manning, C. (2001). Distributional phrase structure induction. In Daelemans, W. & Zajac, R. (eds), Proceedings of the Conference on Natural Language Learning (CoNLL) 2001, 113–20. Toulouse: ACL.Google Scholar
Klein, D. & Manning, C. (2002). Natural language grammar induction using a constituent-context model. In Dietterich, T. G., Becker, S. & Ghahramani, Z. (eds), Advances in neural information processing systems 14, 35–42. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Küntay, A. & Slobin, D. (1996). Listening to a Turkish mother: Some puzzles for acquisition. In Slobin, D., Gerhardt, J. (eds), Social interaction, social context, and language: Essays in honor of Susan Ervin-Tripp, 265–86. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Lee, L. (1999). Measures of distributional similarity. In Proceedings of the 37th ACL, 25–32, College Park, MD: ACL.Google Scholar
Lieven, E., Pine, J. & Baldwin, G. (1997). Lexically-based learning and early grammatical development. Journal of Child Language 24, 187–219.CrossRefGoogle ScholarPubMed
MacWhinney, B. (1995). The CHILDES Project: Tools for analyzing talk. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
MacWhinney, B. (2000). The CHILDES Project: Tools for analyzing talk. Volume 1: Transcription format and programs. Volume 2: The Database. Mahwah, NJ: Erlbaum.Google Scholar
MacWhinney, B. & Snow, C. (1985). The Child Language Exchange System. Journal of Computational Linguistics 12, 271–96.Google ScholarPubMed
Nelson, K. (1977). Facilitating children's syntax acquisition. Developmental Psychology 13, 101–107.CrossRefGoogle Scholar
Onnis, L., Waterfall, H. & Edelman, S. (2008). Learn locally, act globally: Learning language from variation set cues. Cognition 109, 423–30.CrossRefGoogle ScholarPubMed
Pereira, F., Tishby, N. & Lee, L. (1993). Distributional clustering of English words. In Meeting of the Association for Computational Linguistics (ACL), 183–90. ACL.CrossRefGoogle Scholar
Pickering, M. & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences 27, 169–225.CrossRefGoogle Scholar
Redington, M., Chater, N. & Finch, S. (1998). Distributional information: A powerful cue for acquiring syntactic categories. Cognitive Science 22, 425–69.CrossRefGoogle Scholar
Ristad, E. & Yianilos, P. (1998). Learning string edit distance. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 522–32.CrossRefGoogle Scholar
Sachs, J. (1983). Talking about the there and then: The emergence of displaced reference in parent–child discourse. In Nelson, K. E. (ed.), Children's language, Vol. 4, 1–28. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Schütze, C. (1996). The empirical base of linguistics: Grammaticality judgments and linguistic methodology. Chicago, IL: University of Chicago Press.Google Scholar
Sokolov, J. (1993). A local contingency analysis of the fine-tuning hypothesis. Developmental Psychology 29, 1008–1023.CrossRefGoogle Scholar
Sokolov, J. & MacWhinney, B. (1990). The CHIP framework: Automatic coding and analysis of parent–child conversational interaction. Behavior Research Methods, Instruments & Computers 2, 151–61.CrossRefGoogle Scholar
Solan, Z., Horn, D., Ruppin, E. & Edelman, S. (2005). Unsupervised learning of natural languages. Proceedings of the National Academy of Science 102, 11629–11634.CrossRefGoogle ScholarPubMed
Stine, E. & IIIBohannon, J. (1983). Imitations, interactions, and language acquisition. Journal of Child Language 10, 589–603.CrossRefGoogle Scholar
Stolcke, A. & Omohundro, S. (1994). Inducing probabilistic grammars by Bayesian model merging. In Carrasco, R. C. & Oncina, J. (eds), Grammatical inference and applications, 106–118. Berlin: Springer.CrossRefGoogle Scholar
Suppes, P. (1974). The semantics of children's language. American Psychologist 29, 103–114.CrossRefGoogle Scholar
Szmrecsanyi, B. (2005). Language users as creatures of habit: A corpus based analysis of persistence in spoken English. Corpus Linguistics and Linguistic Theory 1, 113–49.CrossRefGoogle Scholar
Valiant, L. (1984). A theory of the learnable. Communications of the ACM 27, 1134–1142.CrossRefGoogle Scholar
Waterfall, H. (2006). A little change is a good thing: Feature theory, language acquisition and variation sets. Unpublished doctoral dissertation, University of Chicago.Google Scholar
Waterfall, H. (submitted). Relation of variation sets to noun and verb development. Manuscript submitted for publication.Google Scholar
Wolff, J. (1988). Learning syntax and meanings through optimization and distributional analysis. In Levy, Y., Schlesinger, I. M. & Braine, M. D. S. (eds), Categories and processes in language acquisition, 179–215. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar