Lila R. Gleitman - US grants

ABSTRACT This proposal from the University of Pennsylvania requests funds to establish a Science Technology Center for Research in Cognitive Science. The Director of the Center will be Professor Aravind K. Joshi. The Center for Research in Cognitive Science unites a diverse and richly interconnected group from many traditional disciplines (computer science, linguistics, mathematics, philosophy, and psychology). The goal of the research is to understand the processes and mechanisms by which human beings acquire knowledge about their environment, store and retrieve that knowledge, communicate it to others, and apply it to carry out actions and manipulate their environment. The research is organized into three separate but highly interrelated themes: perception and action, language learning, and language processing. Research in the area of perception and action spans the processes involved in the first stages of visual and auditory representation of spatial and spectral information, to higher order representations of more complex attributes, to the storage and retrieval of such representations by the organism as they are used in goal-oriented actions. The study of language learning focuses on how children develop the abstract representations of language on the basis of their visual and auditory perceptions. The research in language processing combines investigation of formal systems with investigation of computational models, all in the context of empirical study of a wide range of natural languages. Significant features of the perception and action research are its increasing fidelity to actual neural computation and its sophisticated computational modeling and related potential for contributing to artificial intelligence technology. The language learning research has significant potential for technological spin-off in machine learning and automatic acquisition of lexical and grammatical information for language systems, crucial to the development of grammars sufficient for the robust analysis of unconstrained text. And the language processing research will have significant impact on the technological base for human- computer interaction, in particular the design of natural language interfaces for data base and expert systems and knowledge-rich systems in general. This Center will stimulate enhanced activity in precollege education and in the development of human resources.

This Integrative Graduate Education and Research Training (IGERT) award supports a multidisciplinary graduate training program at the University of Pennsylvania designed to integrate the computational, cognitive and neuroscientific study of communication and communication systems, be they characterized as human-linguistic, animal or machine. The primary purpose is to create a new breed of communication scientists capable of integrating theoretical issues, methods, and formalisms that are currently distributed across graduate programs as diverse as anthropology, biology, computer science and engineering, linguistics, neuroscience, philosophy, and psychology. The intellectual merit consists of the two interrelated research themes that will unite and guide graduate training. The first theme emphasizes communication as a dynamical process, one that unfolds along multiple time scales varying from milliseconds (as in planning and understanding speech) to centuries (as in evolving dialects, languages, and systems of animal communication). The second theme emphasizes communication as a context-sensitive process, where contexts range from the physical setting and communicative history of a specific conversation, to the linguistic, social and technological assumptions of social groups. Trainees will be co-advised by a multidisciplinary faculty team and will commit to a five-year graduate training program, consisting of: (1) core disciplinary training in one of the current graduate programs above; (2) one-year cross-disciplinary training in a chosen second discipline, including completion of a publishable research project; (3) participation in a weekly interdisciplinary research meeting throughout the 5-year program; and (4) completion of an advanced course in the mathematical foundations of communication specifically designed for this program. Broader impacts of this program include applications in industry, technology, and clinical settings. IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries.

DESCRIPTION (provided by applicant): This project aims to discover how languages are learned and understood at the levels of both word and sentence. The first problem we address is understanding how learners find out what words such as "dog" and "cat" signify. The second problem is learning how these words combine semantically in sentences such as "The dog bites the cat" versus "The cat bites the dog." Though the words in this case are all the same ones, in English their ordering (more precisely, the structure that binds them) determines their semantic roles (do-er or done-to) with respect to the "biting" act. Some languages rarely use this ordering method and if they do their orders may be different, so learners of English have to acquire these properties by analyzing the speech that they hear from adults to derive the English facts. In most cases children don't get explicit instruction about most of the word meanings or about the syntax, but they learn even so. The learning process applies not only to children but also to learning of second languages by older individuals, including adults. Much of the work proposed herein uses a relatively new experimental technique, developed in our laboratories under earlier funding of this grant, in which children's eye gaze is tracked as they hear spoken descriptions of the surrounding visual world. Specifically, children hear instructions that require them to make an implicit choice about the intended structural organization of ambiguous utterances such as "I saw the man with a telescope." By manipulating potentially informative cues to the intended structure (e.g., verb information, prosodic (tune) information and situational/discourse cues), children's eye gaze and other behaviors can reveal their sensitivity to and representation of these information sources. In the upcoming funding period, we propose: (a) To expand and test our developmental account of how children learn to recover the grammatical properties of a sentence as it is heard, by examining eye gaze responses to ambiguous sentences at different ages;(b) To explore how multiple linguistic and non-linguistic cues regarding speaker's intentions are used by the child to uncover word and sentence meaning;(c) To examine what is tracked by the child regarding the meaning of verbs and other relational lexical items as they hear them;(d) To examine how sentences understanding procedures are learned and used in languages that are quite different from English in the clues to meaning that they offer (specifically, Korean, Tagalog, and Kannada, and perhaps two others). The potential applications of these findings to education are significant, as vocabulary and sentence understanding skills are fundamental to successful functioning in the technological culture of the 21st century, and many children are in need of enhancing and remedial intervention. In addition, as the United States citizenry becomes progressively more multilingual, and is increasingly drawn into global interactions, the ability to acquire second, and even third and fourth, languages becomes an ever more precious social and economic commodity PUBLIC HEALTH RELEVANCE This project is designed to further the understanding of how young children learn what the words in their language mean, and how these words are combined to make meaningful sentences. The ability to understand spoken and written language rapidly and close to errorlessly is a basic requirement for economic and social well-being in 21st Century American life. The findings are expected to be relevant to second language learning as well;multilingualism is an increasingly precious commodity for Americans as they interact more and more with speakers of different languages both with the country and with cultures around the world.

Project Summary The goal of this project is to understand how children (and older second-language learners) discover the meanings of words and their semantic roles in sentences. Even for simple words like dog it is not easy to glean what they mean from observing (as one usually will) that there is a dog in sight. For after all, when a dog is in sight, so is his fur, and his ear, and the joyous wagging of his tail. To which of these factors or properties does the pointing finger refer when an adult says Look at that doggie! Things get even harder when the child has to learn the meanings of words like idea or think because in these cases there's nothing so obvious to point to out there in the world. Yet children of three and four years of age understand and utter such apparently abstract words. Our projects take off from the observation that not all words occur in the same places in sentences, for instance one can say I think (or see) that you're cute but not I jump that you're cute. Strikingly, children as young as two- and three-years of age are sensitive to these positional privileges, which in turn give clues to word meaning. Because it is hard to get information about children's word meanings or their learning by asking them for definitions or the like, we use implicit methods such as tracking children's eye gaze direction and responsiveness to queries that place words into different visuo-social environments and into different syntactic structures to find out about their evolving word knowledge. These issues are of much more than academic interest. No tested property of child cognition or behavior is a better predictor of school and work-place success than vocabulary growth in the first few years of life. Vocabulary scores diverge for children of higher or lower socio-economic status as early as the second birthday, and these differences increase throughout the school years, influencing all the child's subsequent learning. So our work extends to discovering ways that actual home and early school environments can maximize the supportive environment for vocabulary and syntax acquisition. Our past work gives strong indications of what these favorable learning environments are, and the present proposal inquires more deeply into these factors.