Howard Poizner - US grants

ABSTRACT This project is concerned with the study of visual gestural languages, primary linguistic systems that are not derivative from spoken language. The existence of such fully expressive systems arising outside the mainstream of spoken languages affords a new vantage point for investigating biological constraints on linguistic form. The research centers around the structured use of space and movement in the grammatical processes of signed languages. The investigator couples new linguistic analyses with powerful techniques for three dimensional computergraphic analysis in a series of experiments on the neurological control of language and movement. The experiments probe signers with deficits at different levels of linguistic representation and motor control, examining the neurological control of language and movement through the quantitative analysis of the breakdown of both. The convergence of formal linguistic, objective three- dimensional measurement, and neurological approaches will provide unique information about the functional anatomy of language itself.

This proposal investigates the nature of the neural basis of motor behavior, as a special window into higher brain functions. In this investigation, we link sophisticated three-dimensional computergraphic analyses of movement with experiments that allow us to infer underlying motor control processes performed under conditions of failure of specific motor systems. All of the experiments that follow proceed from this unique vantage point, and should mark a significant advance in our understanding of brain function for motor behavior. Five experiments are proposed that investigate performance of patients with damage to three central motor systems of the brain. Neural Basis of Motor Planning and Control Experiment 1-3 study the neural basis of motor planning and control, beginning with the possible breakdown of a motor law, moving to the spatial control of hand trajectories, and finally to underlying brain processes for complex movements. Neural Basis of Motor Equivalence Experiment 4 investigates the neural basis of a process of motor control integral to the production of speech as well as control of the limbs, that of motor equivalence. Interplay Between Linguistic and Motor Behavior Experiment 5 begins the investigation of the interplay between neural control processes for linguistic and for motor behavior, from the study of a motor disorder in deaf signers. These three-dimensional, computergraphic analyses of movement should not only advance our understanding of the neural basis of motor behavior but should also serve as a useful tool in evaluating diseases which affect the motor systems of the brain.

DESCRIPTION (Investigator's Abstract): New techniques for three- dimensional motion analysis are utilized to analyze the mechanisms of two complementary disorders affecting the control of movement that until now have been studied primarily only with conventional means: patients with left-hemisphere lesions and limb apraxia, who show deficits in planning complex motor acts, and patients with right hemisphere lesions and neglect, who show deficits in motor intention, spatial attention, and in the representation of extra-personal space. Differential deficits in the organization of reaching movements is investigated, experimentally manipulating requirements for movement planning, sensorimotor transformations, and movement execution. Patterns of pointing errors in 3D space, distortions in 3D hand trajectories, and deficits in inter-joint coordination during the movements are analyzed. Experiments utilize a programmable robot arm to present targets in known locations in one of two planes in space. Spatial Perception Thresholds. Pure perceptual errors will be measured by having subjects visually discriminate actual target locations from nearby locations presented I-I sec earlier and then extinguished. Pointing to Actual and Memorized Targets With and Without Visual Control. Disturbances in pointing movements are investigated when subjects use different conditions of visual feedback: no visual feedback during the movement, vision of the arm but not of the target, or vision of the arm and the target. Deficits in sensorimotor transformations, trajectory planning, and in spatial attention underlying action will be investigated. Pointing to Memorized Targets Under Conditions of Extreme Head Rotations. The degree to which hemispatial neglect is governed by head or body position relative to external space is investigated by having subjects turn (heir heads to the extreme right or to the extreme left after target presentation but before pointing is initiated. Right-lesioned subjects with neglect may show an asymmetric contraction of space when pointing with their heads rotated to the extreme right. Planning Trajectories to Avoid an Obstacle. Deficits in motor planning will be investigated in experiments in which subjects point to remembered target location with and without visual feedback, in the presence of an obstacle. The position of the obstacle with respect to the initial arm position and target location will be changed systematically. Right-lesioned subjects may exhibit specific distortions in the relation between the two subcomponents of the movement pointing itself and avoiding the obstacle. Performance of left-lesioned apraxic subjects on the pointing tasks will allow insight into the degree to whim left hemisphere movement control systems specifically mediate learned, skilled movement versus kinematic planning and sensorimotor transformations for movement in general.

biomedical equipment purchase;

ABSTRACT Manual communication, with its very different motoric and perceptual substrate, offers a fresh and exciting opportunity to investigate the biological foundations of language. In a series of experiments, we explore the nature of American Sign Language (ASL) and its neural substrate. Biological and Linguistic constraints on Structure. We investigate the ways in which requirements of production constrain the 'phonological' structure of language, sorting natural constraints imposed by the visual gestural modality from language-particular characteristics. We also initiate new analyses of ASL verbs, which are the blueprint for and the backbone of the ASL sentence. The research aims toward a comprehensive and systematic account of verb class organization, lexical representation, and morphosyntax in ASL. Neurological Control of Language and Movement. The study of sign language probes the interaction of the brain's control of language and of movement. We combine new techniques of three-dimensional computer-graphic movement analyses with linguistic analyses to illuminate neural mechanisms controlling language and movement from an important new perspective: We contrast brain damaged signers with deficits at different levels of linguistic processing and motor control. Biological and Linguistic Constraints on Perception and Action. Signs in American Sign Language are not static, but rather move in complex, highly patterned dynamic arrays. We quantitatively investigate the ability of signers with motor disorders to analyze various types of visual motion stimuli, ranging from the simplest analysis of velocity, to motion with linguistic content. Moreover, we investigate the processing stages underlying their ability to point to memorized target locations in 3-D space. These data sets will help uncover the perceptual-motor underpinnings of the neural substrate for sign language. The convergence of formal linguistic, objective three- dimensional measurement, and neurological approaches in this project will provide fresh perspectives on issues in brain organization for language that are relevant for all natural languages, both spoken and signed.

Nicaraguan Sign Language arose fifteen years ago when hundreds of previously isolated deaf individuals were brought together in schools for the first time as the result of an educational reform movement dedicated to providing literacy training and a fourth-grade education to everyone. For virtually the first time, it became possible to directly observe the emergence of a new language. This case is particularly interesting because this language arose not as the result of language contact or the creolization of previously existing languages, but rather from the merging of idiosyncratic gesture systems, called "home signs" that were used for communication within the immediate family by the first generation of deaf children to enter the schools. Home signers quickly began to share their idiosyncratic system imparting to the communication system more and more conventionality. In one generation, young children exposed to a mix of gesturing began to produce a communication form radically different from their input. The new form was more fluid, more complex and wholly language-like. With no co-existing signed languages and limited access to Spanish as a result of low literacy and inability to hear, the only source from which this highly complex linguistic structure could have come was the human brains of those very young first language learners. This constitutes the cleanest case to date in support of the proposal that humans are innately endowed with the capacity to acquire language--even when the input is non-optimal. This project will explore the rich source of evidence for innate language capacity offered by the Nicaraguan case. It begins with a systematic archiving, transcription and analysis of the language data collected over the past ten years including vocabulary, sentence structure, free conversation and controlled narratives elicited via 1.5 minute nonverbal cartoons. These cartoons are the stimuli for a population study that will document the signi ng of the entire population of signers in Nicaragua. The research involves several components: 1. the population study where a variety of factors conditioning sign fluency are considered including age, chronological year, and amount of exposure to signing; 2. the issue of critical mass (i.e., "How many gesturers does it take to make a language?); 3. the process by which a language emerges. While addressing issues of critical period, critical mass and the innate contribution of the human brain to language, the project will also produce a systematic linguistic analysis of Nicaraguan Sign Language. The areas of focus include the "phonetics" of this new language, an indepth analysis of its classifier and coreference systems, and the coding and analysis of the use of nonmanual gestures and facial expressions marking questions, topics, and a host of other grammatical functions.

DESCRIPTION (Adapted from Applicant's Abstract): This proposal undertakes the first systematic motion analyses of the performance of subjects with Parkinson's Disease (PD) in executing unconstrained, multi-joint 3D reaching movements. The focus is on uncovering the role that the basal ganglia may play in integrating sensory information from different sources into the motor plan. Three-dimensional pointing errors, movement kinematics, and interjoint coordination will be analyzed. Experiments utilize a programmable robot arm to present targets in known locations in one of two planes in space. Subjects will perform movements under a range of conditions which vary the available sensory information, memory demands, and task constraints. Spatial Perception Thresholds. Pure perceptual errors will be measured by having subjects' visually discriminate actual target locations from nearby locations presented 1-2 sec earlier and then extinguished. Pointing to Visually Defined Actual and Memorized Targets With and Without Vision of the Moving Arm. Disturbances in pointing movements to targets performed in darkened room will be investigated under different conditions of visual feedback: no visual feedback during the movement; vision of the moving fingertip but not of the target; vision of the target but not of the finger; vision of the fingertip and the target. Pointing to Memorized Targets Defined Proprioceptively. Two modes of proprioceptive target presentation will be used that will allow or prevent the subject from having access to central control signals underlying the arm movement. Pointing to Memorized Targets Defined Cutaneously. Disturbances in pointing movements to somatosensory targets also will be analyzed and compared to disturbances for visually or proprioceptively presented targets. Movement Speed Regulation: Peripheral Deficits versus Central Strategy Adaptation. Movements to memorize targets performed at different speeds with and without vision of the arm will be compared. Comparison Across Patient Groups. Finally, data on Parkinsonian subjects will be compared with separate data acquired on subjects lacking proprioceptive information from their limbs as well as subjects with unilateral cortical lesions. This combination of movement analyses with behavioral manipulations allows characterization of deficient motor performance and the ability to dissect out those elements of sensorimotor processing which may be most impaired in Parkinsonism.

DESCRIPTION (provided by applicant): Our findings in the current grant period have led us to hypothesize that a major difficulty for patients with Parkinson disease (PD) is in assembling and using new sensorimotor mappings or coordinations. These process play a major role both in ongoing motor performance and in the acquisition of new skills, in addition, our preliminary data are consistent with a general observation that these processes may be relatively resistant to current therapeutic modalities. Furthering our understanding of this deficit, examining its impact on motor learning, and investigating the ability of dopaminergic therapy to reverse this deficit are the guiding aims of this proposal. The present proposal presents three experiments that are designed to confirm and extend our hypothesis and to investigate the degree to which dopaminergic therapy is able to remediate these deficits. The first two experiments (Specific Aims 1 and 2) introduce the requirement that subjects learn to move within a virtual environment as a prerequisite to establishing the new sensorimotor coordinations necessary for accurate target acquisition. We require subjects to master distortions which create discrepancies between the apparent (virtual) and real (proprioceptively signaled) location of their arms and to generalize the resulting learning to untrained regions of this environment. By dissociating movements from their normal sensory correspondences, we will challenge subjects' abilities to reconfigure their sensorimotor coordinations. The third experiment (Specific Aim 3) challenges patients by requiring them to integrate different motor acts in order to acquire visually-presented, real targets by compensating for a mechanical perturbation of the trunk during a trunk-assisted reach. We have integrated and coupled our previously developed system for analysis and display of three dimensional movements with our newly developed virtual reality environment. We will examine not only subjects' accuracy, but also the path, timing, and structure of their movements under different conditions and types of imposed distortions, in order to measure both performance and learning when PD patients are OFF versus ON dopaminergic therapy. By contrasting the performance and capacities of PD patients on and off dopaminergic therapy to that of comparable normals, we can both obtain clues as to how to overcome PD dysfunction and gain an insight into the key role of the basal ganglia in movement.

Human infants must learn complex skills to interact effectively with parents and other humans, but these social skills emerge at somewhat different ages in different infants. How can we explain this variability? How do infants attend to their social world, and thereby learn routines to interact effectively with other people? This project follows a group of 45 healthy toddlers who have been tested extensively from 3 to 18 months of age on a variety of changing cognitive and emotional responses to social stimuli. The same infants have been observed regularly at home in interactions with their parents. The current project asks how these toddlers' emerging social skills reflect their individual differences in cognition and emotion as infants, and on the different social input provided by their parents. The project focuses on changes in language and imitation skills from 18 to 24 months of age, and the brain dynamics that underlie these skills. The toddlers who were tested and observed starting at 3 months of age will be invited to participate again at 20 to 24 months of age. New sessions will use a unique system at UC San Diego: a Mobile Brain Dynamics (MoBI) facility for recording EEG (electroencephalographic) and body motion-tracking data simultaneously from two people. The project will use this system to record toddlers and parents as they engage in three types of interactions: 1) toddlers following parent's pointing (or line-of-gaze), 2) toddlers reacting to words spoken by parents, and 3) toddlers imitating parents' simple actions. These interactions represent important social achievements for toddlers. Advanced EEG analysis will be performed on electrical potentials measured on toddlers' and parents' scalps. At the same time special cameras will record the positions of their heads and arms. This design will therefore yield a continuous record of changes in the toddlers' and parents' brain electrophysiology (reflecting their thinking and emotional reactions) and body positions as they interact. In addition, toddlers will complete a battery of behavioral and language tests. This project will pioneer a new paradigm for studying the social development of young children, and yield the most complex and complete data available on how early social-attention behaviors relate to early language and imitation, and brain processes underlying these relations. The results will have implications for early childhood education, treatment of developmental disabilities, and parenting practices.

de Sa
0756828

Brain computer interfaces (BCIs) translate basic mental commands into computer-mediated actions. BCIs allow the user to bypass the peripheral motor system and interact with the world directly through brain activity. These systems are being developed to aid users with motor deficits which can stem from: neurodegenerative disease (such as Lou Gehrig's disease, or ALS), injury (such as spinal cord injury), or even environmental restrictions which make movement difficult or impossible (such as astronauts in space suits). BCI systems typically require extensive user training to generate reproducible and distinct brain waves. Furthermore, until very recently, most BCI systems have interacted with the user in unintuitive or unnatural ways, such as moving a cursor or bar left and right by engaging in two unrelated forms of mental imagery, such as moving the right hand vs. the left foot. Realistic visual feedback of interpreted motor action should substantially improve usability and performance of BCI systems. This hypothesis is based on four observations: 1) humans have evolved to adapt their motor control in response to visual and proprioceptive feedback; 2) rapid motor adaptation is demonstrated in virtual reality experiments; 3) animals improve their neural signal when given visual feedback of their decoded neural activity; and 4) visual feedback of interpreted movement should activate the mirror neuron system, producing a stronger movement signal. The proposed work aims to improve upon current BCI systems based on motor imagery by providing more natural and lifelike feedback. This task can be broken down into 3 main objectives: 1) analyze motor imagery with visual feedback in an offline setting; 2) develop algorithms for real-time EEG analysis; and 3) construct a real-time BCI system utilizing lifelike motion animations as visual feedback. While results of objectives 1 and 2 should each in their own right contribute to the current state of the art in BCI systems, the largest BCI performance and usability gains should be made by introducing lifelike feedback into an online paradigm in the third objective. The proposed system can also be used to study learning and sensory-motor processing in normal subjects by studying their adaptation to the system. It may also inform more costly invasive recording experiments by helping to determine optimal placements of implants. All software written for EEG signal processing and analysis will be made available as add-ons to EEGLAB which is distributed in accordance with University of California policy for research, education, and non-profit purposes. The EEGLAB project is also developing an EEG database in conjunction with the San Diego Supercomputer Center. Representative data sets will be released via this database in accordance with University of California policy.

Intellectual Merit: This project aims at combining cognitive and computational neuroscience, neuroengineering and system identification towards a transformative understanding of the way distributed brain dynamics interact with motor activity in humans. 3-D body and limbs movement kinematics, eye movements and electroencephalographic (EEG) spatiotemporal brain data will be recorded simultaneously during motor control and adaptation in healthy and Parkinson?s disease patients. In particular, altered and real world motor tasks will be simulated in 3-D immersive virtual reality technology with force feedback robots providing proprioceptive interaction and feedback. Cognitive, behavioral and kinematics data will constrain the design of large-scale computational models of motor control and adaptation based on known anatomy and physiology of the basal ganglia. Neuromorphic engineering will guide the design of mobile embedded computational systems for real-time emulation of the brain-body models and closed-loop sensory-motor control for Parkinson?s patients. We expect that the development of new machines for neuro-rehabilitation will result in a threefold synergetic interaction between engineering and neuroscience: human-machine interactions will transform the notion of movement control and provide new contexts to study embodied cognition that will benefit neuroscience; in turn, new knowledge in neuroscience and motor control will accelerate the development of adaptive machines for rehabilitation and/or enhancement. Finally, comprehensive and predictive mathematical models of motor control implemented in neuromorphic hardware are expected to lead to new intelligent neuroprosthetic tools.

Broader Impact: Outcomes of this research will contribute to the system-level understanding of humanmachine interactions and motor learning and control in real world environments for humans, and will lead to the development of a new generation of wireless brain and body activity sensors and adaptive prosthetics devices. This will advance our knowledge of human-machine interactions, stimulate the engineering of new brain/body sensors and actuators, and have a direct influence in diverse areas where humans are coupled with machines, such as brain-machine interfaces, prosthetics and telemanipulation. We anticipate that the confluence of cognitive and computational neuroscience, control theory and wearable, unobtrusive bioinstrumentation will provide novel non-invasive approaches or the treatment and neuro-rehabilitation of Parkinson?s disease and will potentially transform our understanding of brain/body interactions. The project draws graduate and undergraduate students across divisions and in the NSF Temporal Dynamics of Learning Center (TDLC) and Institute of Neural Computation (INC) at UCSD participating in interdisciplinary engineering and neuroscience aspects of the design, optimization, and training of largescale neuromorphic systems and their human interfaces. Through outreach channels on campus supported by the TDLC and the NSF Research Experience for Undergraduates (REU), the program will actively pursue increased participation in research and education of the next generation of scientists and engineers.