Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-26T18:35:33.227Z Has data issue: false hasContentIssue false
  • English
  • Français

Learning Simple Things: A Connectionist Learning Problem from Various Perspectives

Published online by Cambridge University Press:  28 February 2022

Edward P. Stabler Jr.
Edward P. Stabler Jr.*
Affiliation:
University of Western Ontario
Get access Rights & Permissions [Opens in a new window]

Extract

The performance of a connectionist learning system on a simple problem has been described by Hinton and is briefly reviewed here: a finite function is learned, and the system generalizes correctly from partial information by finding simple “features” of the environment. For comparison, a very similar problem is formulated in the Gold paradigm of discrete learning functions. Identification in the limit from positive text of a large class of functions including Hinton's is achievable with a trivial, conservative learning strategy. Using Valiant's approach, we place an arbitrary finite bound on function complexity and then we can guarantee text and resource efficiency relative to a probabilistic criterion of success. But the connectionist system generalizes. That is, it uses a non-conservative learning strategy. We define a simple, non-conservative strategy that also generalizes like the connectionist system, finding simple “features” of the environment.

Type
Part XIII. Formal Learning Theory
Information
PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association , Volume 1988 , Issue 2: Volume Two: Symposia and Invited Papers 1988 , 1988 , pp. 424 - 441
Copyright
Copyright © 1989 by the Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

1

I am grateful to Geoffrey Hinton, Edward Stabler, William Demopoulos, Daniel Osherson, and Zenon Pylyshyn for helpful discussions of this material. This research was supported in part by the Canadian Institute for Advanced Research and the Natural Science and Engineering Research Council of Canada.

References

Brayton, R.K. and McMullin, C. (1982), “The Decomposition and Factorization of Boolean Expressions.” Procs. 1982 IEEE Int. Symp. on Circuits and Systems: 49-54.Google Scholar
de Micheli, G., Brayton, R.K. and Sangiovanni-Vincentelli, A. (1985), “Optimal State Assignment for Finite State Machines.” IEEE Transactions on Computer-Aided Design, CAD-4(3): 269–285.CrossRefGoogle Scholar
Gold, E.M. (1967), “Language Identification in the Limit.” Information and Control 10: 447–474.CrossRefGoogle Scholar
Goodman, N. (1955), Fact, Fiction and Forecast. Cambridge, Massachusetts: Harvard University Press.Google Scholar
Hartmanis, J. and Stearns, R.E. (1966), Algebraic Structure Theory of Sequential Machines. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Hinton, G.E. (1986), “Learning Distributed Representations of Concepts.” Procs. of the 8th Ann. Conf. of the Cognitive Science Society: 1-12. New Jersey: Erlbaum.Google Scholar
Hinton, G.E. (1987ms), “Connectionist Learning Procedures.” Forthcoming in Artificial Intelligence.Google Scholar
Johnson, R.A. and Wichern, D.W. (1988), Applied Multivariate Statistical Analysis, 2nd Edition. Englewood Cliffs: Prentice-Hall.Google Scholar
Osherson, D.N. (1978), “Three Conditions on Conceptual Naturalness.” Cognition 6: 263–289.CrossRefGoogle Scholar
Osherson, D.N., Stob, M. and Weinstein, S. (1986), Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. Cambridge, Massachusetts: MIT Press.Google Scholar
Pinker, S. and Prince, A. (1988), “On Language and Connectionism: Analysis of a Distributed Model of Language Acquisition.” Cognition 28: 73–193.CrossRefGoogle ScholarPubMed
Pylyshyn, Z.W. (1988ms) “The Role of Cognitive Architecture in Theories of Cognition.” Forthcoming in VanLehn, K., ed., Architectures for Intelligence.Google Scholar
Quine, W.V. (1952), “The Problem of Simplifying Truth Functions.” American Mathematical Monthly, 59: 521–531.CrossRefGoogle Scholar
Savage, J.E. (1976), The Complexity of Computing. Toronto: Wiley.Google Scholar
Stabler, E.P. Jr. (1989), “Learning Simple Things: A Connectionist Learning Problem from Various Perspectives.” Technical Report, Department of Computer Science, University of Western Ontario.Google Scholar
Valiant, L.G. (1984), “A Theory of the Learnable.” Communications of the ACM 27:1134–1142.CrossRefGoogle Scholar