Bayesian Model of Language Learning

Language Acquisition

Language learning has been a huge challenge in cognitive science. Language is complex, yet children learn their native language rapidly and robustly without supervision.

But after many attempts, there are still very few working models of language learning which can learn all aspects of a language.

This Model of Language Learning

This website demonstrates a model of language learning. The model can learn all aspects of a language (word sounds, meaning and grammar) rapidly, starting with no linguistic knowledge. It learns a set of 40 English words from 800 learning examples; the words can then be used to understand or produce a large set of English sentences.

This is a very capable model of language learning, for one reason: It is based on simple mathematical operations of Bayesian inference and learning; so the model can be proven to work.

Foundations of the Model

The model combines two established frameworks in cognitive science. These are (1) Cognitive Linguistics - in which language is closely related to other cognitive faculties in the brain - and (2) Bayesian Cognition - in which brains apply Bayes' theorem to find the most likely interpretation of events around them.

These ideas are combined in a simple mathematical framework, which underpins the fast and robust learning of the model.

A Theorem of Language Learning

A language is modelled as feature structures, with two complementary operations on them - unification for language use, and generalisation for language learning. The following theorem can be proved:

If speakers produce language by unifying word feature structures, and children learn words from examples by generalising them, then by this process, the feature structure for any word is replicated faithfully through the generations.

This theorem is a proof that the learning model works.

The theorem shows how diverse languages can persist over many generations, and implies that the model can learn any language. The faithful replication of words is to language, as DNA replication is to life

The Language Controversy

There is a long-running controversy in the study of language, between theories of Generative Grammar (following Noam Chomsky), and work in Cognitive Linguistics, such as this model.

A central argument made for generative grammar is the argument of the Poverty of the Stimulus. This holds that learning a first language from a limited learning stimulus is so difficult, that important parts of grammar must have been specified innately in the human brain.

This learning model shows that a complex language, including its grammar, can be robustly learnt from small numbers of learning examples. So there is no need for grammar to be innate. This counters the argument of the poverty of the stimulus.

Downloads and Enquiries

You can download the following materials from this site:

- Three papers which describe the model, its foundations, and the learning theorem

- A slideset summarising the papers

- The learning program, with instructions to run it

Please send comments and questions to