David Touretzky - US grants

Massively parallel connectionist systems having distributed representations of knowledge, have two fundamental problems: developing faster and more effective learning procedures for connectionist networks, and developing techniques by which such networks can handle complex symbolic knowledge structures in addition to the lower-level sensory knowledge being studied by groups. The learning work generally focuses primarily on variations of the existing so-called Boltzmann and back-propagation procedures. In this proposed effort "variable plasticity" techniques in which not all of the weights have the same ability to change during learning. The learning procedures developed will be evaluated by application to problems in speech understanding, low-level image processing, and control of a manipulator. Preliminary experiments in these areas are described in the main proposal. The work on symbolic representations will focus on language understanding and commonsense reasoning, specifically: increasing the subtlety and richness of distributed symbolic representations, combining multiple sources of syntactic and semantic constraint via parallel relaxation, investigating problems in matching and complex inference, using learning to adjust the behavior of an adaptive symbol processor problems in artificial intelligence and cognitive psychology. systems. The proposed work is cross-disciplinary in nature, applying techniques from mathematics, computer science and the new field of connectionism to problems in artificial intelligence, cognitive psychology, and

9631336 Touretzky In order to navigate successfully through the world, especially in the absence of visual input (e.g., in the dark), animals must maintain an accurate estimate of the direction in which they are facing. In rodents, the heading estimate is probably updated by a combination of vestibular cues from the inner ear, visual cues, and information about the animal's motor actions. This award supports the development of computer models of portions of the rodent brain involved in representing head direction, such as the anterior thalamic nuclei and the postsubiculum. Based on what is known about the tuning properties of head direction cells, the head direction system appears to exhibit a self- organizing activity pattern, called an attractor, that shifts as the animal's heading changes. Computer modeling can reveal how this attractor system operates. This research promises to yield new insights into how the nervous system organizes animal behavior and how information is represented in the mammalian brain.

This award supports the development of a computationally explicit theory of operant conditioning, in which animals determine the effects of their actions and adjust their behavior to maximize reward. The Rescorla-Wagner model of classical conditioning and its various descendants have yielded considerable insight into how associations between sensory stimuli andreflex actions may be acquired. But operant conditioning evokes more complex and deliberate behavior patterns, for which there is no comparable computational model. The theory being developed includes four types of learning: (i) on-line learning of reward predictors based on observed reinforcement contingencies, (ii) acquiring secondary reinforcers, such as the sound of a food dispenser being activated, (iii) generating new actions by selecting and shaping innate behaviors, and (iv) refining perception to focus on task-relevant signal discriminations. In addition to testing the theory with computer simulations of classic animal learning experiments such as the Delayed Match to Sample task, the theory is being embodied in an RWI B21 mobile robot. This research promises a new class of learning robots that can interact with people in much the same way that animals interact with their human trainers

This project is being funded through the Learning and Intelligent Systems (LIS) initiative. It is concerned with learning that is incremental in nature, resulting in skills that improve over time. Such learning is responsible for the development of perceptual discriminations, stimulus associations, and motor skills, and is seen in both humans and animals. The goal of the research is a systems-level neural theory of incremental learning. The project combines computer simulations with neurophysiological recording from the brains of behaving rats and monkeys, functional magnetic resonance imaging in humans, and robotic implementation. It is known that many parts of the mammalian brain contribute to incremental learning. Although the cortex may play a central role, other brain areas known to make vital contributions include the basal ganglia, hippocampus, amygdala, and cerebellum. It is important to understand the roles of these various areas and their interactions with each other during learning. Experimental work to be performed includes recording from various areas of the brains of rats and monkeys during learning and imaging the brains of humans performing analogous tasks. Models of neural function will be developed based on the monkey experiments and will be implemented on a mobile robot. These implemented models will allow the training of the robot, and a comparison of the learning that takes place with that observed in humans and animals. The development of theories of incremental learning will provide a better understanding of how the process occurs and may result in improved approaches to the development of skills in both humans and machines.

The PI will develop a cognitively plausible reinforcement learning (RL) architecture as a model of instrumental learning in animals and robots. Although RL was initially inspired by animal learning phenomena, the field has since developed mainly by addressing AI concerns. A major limitation of current RL architectures as cognitive theories is the representation of state space. Models that maintain explicit state representations (such as Q~tables) are limited to simple domains with only a few variables, while models that represent the state space implicitly (e.g., using a neural net function approximator) require large amounts of training data and unreasonably long training times compared to real animals. The PI's approach is to develop specialized representations of state space that are appropriate for modeling animal behavior and can support desired generalizations. The simulated animal's working memory will encode sensory stimuli, state change events, and the animal's own actions. An explicit state representation would encode the conjunction of all these variables, generating a combinatorial explosion. The proposed alternative approach is for the model to form conjunctions of selected variables, allowing it to incrementally expand its state description while focusing on just those dimensions that are relevant to the task being learned. Heuristics based on fast, single-layer neural net learning will be developed to select useful conjunctions as a function of recent experience. The PI also will investigate matching the current state of working memory with records of entire past states, or episodes, in order to predict reward. A flexible architecture will be developed for representing actions in a parameterized manner (so as to provide infinite variability), and with temporal duration (allowing stimuli and rewards to arrive in the midst of execution). Finally, there will be mechanisms for coping with failure of an action to execute successfully or to produce an expected reward; this will provide the basis for modeling phenomena such as effects of partial reinforcement schedules and increased behavioral variability during extinction. If successful, this work will advance the state of the art of reinforcement learning by introducing new techniques for handling complex state and action spaces. This has important implications for theories of animal cognition, for robots that learn by exploration and experimentation, and for robots intended to learn from human teachers.

9987588
David S. Touretzky - Carnegie Mellon University
IGERT: Cross-Disciplinary Training in the Neural Basis of Cognition

This Integrative Graduate Education and Research Training (IGERT) award supports the establishment of a new multidisciplinary, multi-institution, graduate training program of education and research within the Center for the Neural Basis of Cognition at Carnegie Mellon University and the University of Pittsburgh. The CNBC offers interdisciplinary training to students in a variety of fields who seek to understand how cognitive processes arise from neural mechanisms. Participating departments at Carnegie Mellon are Computer Science, Psychology, Robotics, and Statistics. At the University of Pittsburgh they are Mathematics, Neurobiology, Neuroscience, and Psychology. The IGERT training program is a new kind of cross-over training experience in which students develop professional competence in an area outside their home discipline. For example, a computer science student whose research involves computer modeling of the hippocampus could work in a neurophysiology lab, learning to do parallel multiunit recording from the hippocampus of behaving rats. A psychology student doing functional brain imaging could receive cross-over training in statistical analysis techniques for functional imaging data. During their first year in the IGERT program, students will acquire necessary background knowledge through a combination of coursework, directed reading, and observation in the lab. In the second year students will work half-time in their host lab to complete a small research project of their own.

IGERT is an NSF-wide program intended to meet the challenges of educating Ph.D. scientists and engineers with the multidisciplinary backgrounds and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing new, innovative models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries. In the third year of the program, awards are being made to nineteen institutions for programs that collectively span all areas of science and engineering supported by NSF. The intellectual foci of this specific award reside in the Directorates for Biological Sciences; Computer and Information Science and Engineering; Social, Behavioral, and Economic Sciences; Mathematical and Physical Sciences; and Education and Human Resources.

space perception; memory; mathematical model; learning; neural information processing; hippocampus; brain mapping; model design /development; gene environment interaction; imagery; cell population study; electrophysiology; visual perception; psychological models; neurons; experience; neuropsychology; behavioral /social science research tag; neuropsychological tests; computer simulation; electroencephalography; laboratory rat; visual stimulus; statistics /biometry;

This Integrative Graduate Education and Research Training (IGERT) project will develop new directions for an existing cross-disciplinary training program offered by the Center for the Neural Basis of Cognition (CNBC) at Carnegie Mellon University and the University of Pittsburgh. Eleven doctoral programs are formally affiliated with the program, which trains young scientists in interdisciplinary approaches to understanding how cognitive processes arise from neural mechanisms. One half of the new initiative concerns new technologies for experimental neuroscience, e.g., optical, magnetic resonance, and magnetoencephalographic approaches to functional brain imaging, and analysis of neuronal population activity patterns using advanced statistical and data visualization algorithms. The complementary half concerns the applications of systems and cognitive neuroscience to technology development, in areas such as neural prostheses, neural control, and cognitive robotics. The most successful aspects of the current CNBC IGERT program will be retained: a common core curriculum, use of multiple advisors from different disciplines, and a competitive proposal process for selecting students for funding.
The project will train a new breed of scientists to achieve the fullest possible integration of new technologies into cognitive and systems neuroscience research. It will also complement a new initiative at Carnegie Mellon, in collaboration with Spelman College, to establish undergraduate courses in cognitive robotics at several historically black colleges and universities, thereby increasing the pool of minority applicants to graduate programs such as the CNBC program. IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries.

Spelman College together with Carnegie Mellon University propose an integrated set of three educational programs to broaden the participation of African Americans in computing. The first is a series of two-week Computer And Robotics Education (CARE) summer camps for African American middle school students, with follow-up activities during the school year. The second is an annual CARE Computer Olympiad for African American undergraduates, which will build on successful Olympiads already held at Spelman. This program will expand into a series of regional fall Olympiads followed by a national Olympiad in the spring. The third is a CARE Robotics course in "cognitive robotics" to be jointly developed with Carnegie Mellon University. The course will jump start robotics education at HBCU's by providing state of the art robotics equipment (Sony AIBO robot dogs), lecture notes and other curriculum materials, along with the necessary faculty training and technical support. The course will use the Tekkotsu software framework under development at Carnegie Mellon. One-day Tekkotsu workshops will also be offered during the National Olympiads to familiarize additional HBCU faculty and selected students with this new technology. Both the Olympiad competition and the course will be disseminated to other HBCU's over the course of the proposed grant.

Spelman College proposes the ARTSI (Advancing Robotics Technology for Societal Impact) Alliance in collaboration with Florida A&M University, the University of the District of Columbia, Hampton University, Morgan State University, Norfolk State University, Winston-Salem State University, the University of Arkansas-Pine Bluff, Carnegie Mellon University, Georgia Institute of Technology, Brown University, Duke University, the University of Alabama, the University of Washington, and the University of Pittsburgh. Seven of these partners are HBCUs and seven are Carnegie Research I institutions. Their collaboration joins the strengths of HBCUs in conducting outreach and education in a nurturing learning environment with those of the R1's for conducting world class research. The ARTSI Alliance will motivate students to pursue computer science careers by emphasizing the creativity and socially beneficial aspects robotics technology with hands-on projects, curriculum, and media. ARTSI activities will span the academic pipeline from K-12 through the faculty ranks. At the K-12 level, students will be recruited with community outreach using robotics and art, robotics road shows, and a robotics educational film online repository. At the undergraduate level, HBCU students will be exposed to new robotics curriculum, and they will be encouraged to pursue advanced training in graduate school through summer research experiences, collaborative, interdisciplinary robotics projects in the arts and health, instruction in technical film documentation, student virtual film festivals, annual robotics conferences, and instruction in entrepreneurship for computer science. At the faculty level, it will increase the number of HBCU faculty who educate students in robotics and involve students in robotics research by providing faculty mentoring, summer research experiences for underrepresented faculty at R1 robotics labs, robotics summer workshops, and development and dissemination of robotics educational material through a web-based portal. The Alliance will have industry partners, including Seagate, iRobot, Microsoft Research, and Juxtopia, as well as educational partners, including Florida-Georgia Louis Stokes Alliance for Minority Participation and Computer Science Teachers Association.

Computer Science (31)

This project is extending the Tekkotsu open source robot programming framework developed in the PI's lab (see Tekkotsu.org) by creating new primitives for manipulation and for control of posture and balance, and by further enriching the existing repertoire of primitives for vision processing, mapping, and navigation. It also is providing the first systematic study of how a higher level approach to robot programming influences educational outcomes. This project is developing software and course materials that foster a new, higher-level approach to introductory robotics for undergraduates, called "cognitive robotics." Cognitive robotics courses are already offered at Carnegie Mellon, Spelman College, and several other schools with which the PI is collaborating. The project is promoting the wider adoption of cognitive robotics curricula by offering workshops at Carnegie Mellon for computer science educators, making presentations at conferences such as SIGCSE and AAAI, disseminating open source software and educational materials via the web, and creating a cognitive robotics textbook.

Until recently, undergraduate robotics courses have been limited by inexpensive platforms which provide only meager sensors and minimal processing power. Such courses have therefore tended to focus on mechanical construction activities and programming simple reactive behaviors such as wall following. While some platforms provide for an optional video camera, image processing support has typically been limited to crude blob detection, not true computer vision. In cognitive robotics, students use more sophisticated robots that can see and recognize objects, physically manipulate them, build a map of the environment, and navigate on that map. The Sony AIBO robot dog was the first platform suitable for this approach, but other capable platforms are now becoming available. Undergraduates can be taught to program these robots using high-level primitives that draw inspiration from ideas in cognitive science. This allows even beginning roboticists to explore interesting problems in perception and manipulation while the complexities of advanced image processing and motor control are taken care of for them.

Hampton University, in collaboration with Carnegie-Mellon University, Florida Agricultural and Mechanical University, the University of the District of Columbia, Norfolk State University, Winston-Salem State University, Morgan State University, Jackson State University, Elizabeth City State University, Duke University, the University of Alabama Tuscaloosa, and the University of Michigan, proposes the ARTSI Alliance (Advancing Robotics Technology for Societal Impact). ARTSI is a consortium of Historically Black Colleges and Universities (HBCUs) and major research universities (R1s) working together to increase African American participation in computer science, with a focus on robotics. This extension proposal will expand ARTSI to seventeen Historically Black Colleges and Universities (HBCUs) and roughly 10 major research universities (R1s). Hampton University is the new lead institution; Carnegie Mellon University remains the lead R1 school. The extension introduces three new initiatives that (1) improve the quality and uniformity of robotics instruction by developing robotics curriculum modules specific to the needs of HBCUs, (2) pilot a program to attract STEM (Science, Technology, Engineering, and Mathematics) students to HBCUs by offering robot programming activities in local high schools, and (3) pilot skill-building program for rising sophomores to better prepare them to become involved in robotics research. The extension also includes new collaborations with the Caribbean Center for Computing Excellence (a BPC Alliance in Puerto Rico and the US Virgin Islands) and the Defense Advanced Research Projects Agency.

The Rochester institute of Technology -- in collaboration with Auburn University, Carnegie Mellon University, and TERC -- proposes the initial stages of a years-long iterative process to develop an evidence-based, pedagogically-sound computing trajectory for students in middle school through early college. The project is based on a novel three-stage model of programming instruction using the Microsoft Kodu, Alice, and Tekkotsu programming frameworks. It will address the questions:

-- Can the proposed three-stage model for programming instruction, accelerate student progress on mastering computational thinking?
-- Can students develop a deeper understanding of computer science concepts by learning to draw explicit analogies between realizations
of the same idea in different settings, i.e., in different software frameworks, or in Kinesthetic Learning Activities like CS
Unplugged?
-- How can programming environments be made more accessible to students with disabilities such as visual impairment, deaf/hard of
hearing, mobility impairment (e.g., cerebral palsy), or learning and cognitive impairment (e.g., autism or ADD)?

The last question will be addressed in collaboration with Microsoft FUSE Labs on accessibility enhancements to Kodu, and continuing development of BridgIT, an alternative to Alice designed with accessibility in mind.

This planning grant will do preliminary work on these questions, collecting pilot data at a week-long summer camp for students, and a two-day workshop for K-12 teachers. The award will allow the PIs to refine their hypotheses, conduct pilot experiments, deepen their relationships with academic and industry partners, and formulate the detailed research plan needed for the eventual full proposal.