Howard Charles Nusbaum - US grants

This research proposal is describes three projects that are concerned with the structural knowledge and perceptual process that mediate word recognition, perceptual learning of speech, aNd perception of talker characteristics. The first project is concerned with understanding the role of structural knowledge in word perception. Two experiments in this project investigate how structural properties of words are used to segment an utterance into word-length patterns and how these properties may be used to facilitate the process of word recognition. A third experiment is directed at investigating whether the patterns of words in different lexical classes (e.g., open and closed class words) can differentially facilitate recognition of the words in these classes and whether structural differences allow listeners to make word class judgments during recognition, rather than as a result of post-recognition lexical access. The fourth experiment examines how lexical familiarity and frequency recognition of spoken words and the processing of their constituent elements. The second project is concerned with the processes and knowledge that support perceptual learning of speech. The first experiment will investigate how a listener's knowledge of the structural properties of speech may be used to learn to overcome distortions in the phonetic structure of speech. The second experiment is concerned with the role of the organization of the mental lexicon in learning new words. The final experiment will investigate how context-conditioned variability of the pattern structure of a signal affects the perceptual learning and recognition of the information encoded in the signal. Although this experiment will use nonspeech acoustic patterns, the results may suggest how the perceptual system could use the acoustic consequences of coarticulation to facilitate learning and recognition of the acoustic-phonetic structure of speech. The third projects directed at understanding how perception of the characteristics of a talker's voice can affect recognition of the lexical and phonetic content of the talker's speech. The first experiment will investigate possible interactions between the attentional demands of recognizing the phonetic structure of speech and recognizing a particular talker's voice. The second experiment is concerned with the acoustic properties of speech that are used by a listener to focus on a particular talker's voice among other voices and how these properties affect recognition of the information conveyed by that voice. The final experimental will investigate how memory for a particular talker's voice affects perceptual normalization of talker difference to facilitate recognition of the segments in speech. The findings of these three projects haVe broad implications for the diagnosis and treatment of disorders and diseases that affect the perception and comprehension of spoken language.

With support from the National Science Foundation, the University of Chicago Committee on Cognition and Communication will purchase: a video digitizing PowerMac 9500/132, 4 Silicon Graphics Indigo2 ZX workstations, 3 DigitalLinear Tape 20 GB backup systems and a Young Minds CD-ROM recorder. This equipment will permit the group to digitize multimedia recordings of communication situations and to code and analyze various properties of each digitized recording. It will also establish the groundwork for the creation of a large multimedia database of communication situations. The database will contain coded and measured information about each communicative situation. It will include classifications of gestures made qualitatively by researchers acoustical measurements of speech, and measurements of gesture movement using image processing tools. The goal of this research is to investigate the cognitive mechanisms that mediate processing in conversational interaction, and utterance production and comprehension. To fully understand verbal communication, language use must be studied in natural communicative contexts such as conversations, arguments, narratives, recall of emotional events and parent-child interaction. Natural communicative contexts contain rich sources of information that govern the dynamics of language processing in ways that cannot be observed in standard laboratory situations. The analysis system which the Chicago group will employ, will allow studies of this type to be conducted. The instrumentation system will also be used in teaching situations and be available to graduate and undergraduate students.

This grant supports the acquisition of instrumentation to measure Event-Related Potentials (ERPs) in cognitive and social neuroscience research. ERPs reflect the synaptic activity of the brain that occur during mental activity. The requested instrumentation will be used with fMRI brain scanning at the Brain Research Imaging Center (BRIC) at the University of Chicago. BRIC has purchased a 3T scanner to study psychological processes such as spoken language understanding, shifts of attention, perceptuo-motor planning and motor imagery, and social categorization. Although behavioral studies have told us much about these mental processes, these studies cannot indicate the specific way in which the brain implements this processing. fMRI can identify which parts of the brain are involved in processing. Although fMRI can localize cortical areas, blood-flow responses are too slow to measure the timing of activity in these areas. ERPs provide this timing information, but cannot localize brain activity as finely as fMRI. fMRI together with ERP combines high spatial resolution and temporal resolution to investigate the functional processing in the brain. Measuring ERP activity during fMRI will aid in relating temporal and spatial information about cortical processing.

Three broad projects will initially use the instrumentation. One project will investigate interactions between language-specific mechanisms and more general cognitive processes during language comprehension. This research examines how understanding uses attention and memory as well as paralinguistic information. ERP measures will investigate relative timing of cortical activity in working memory and attention compared with specifically linguistic processing. The second project will examine neural mechanisms involved in perception and production of action. Comparison of perceptual processing of symbolic cues for actions with direct human movement cues (to trigger "mirror" cells) will investigate neural mechanisms underlying conceptual operations in motor movements and motor "imagery" circuits. The third project investigates affective evaluation and social categorization addressing the theory that positive and negative attitudes are mediated by different mechanisms. fMRI and ERP will identify relevant brain areas and examine the time-course of processing in affective, perceptual, and autonomic parts of these networks to investigate how affective evaluation systems mediate stereotypic responses in social categorization.

Neoclassical economic theory underlies almost all modern economic policy analyses. Nevertheless, several important deviations from standard assumptions of the theory have been robustly documented in laboratory experiments on economic decision-making. Two such important behavioral deviations are the "endowment effect" and the prevalent display of altruism/social preferences. A multidisciplinary team of researchers from economics, psychology, and neurology will examine the sources of these experimental findings. In this research, the economic decision-making experiments - spanning both laboratory and field treatments - used to detect these deviations will be complemented by neuroimaging techniques to provide insights into the neural mechanisms underlying the behavioral responses. An important hypothesis that will be tested in the context of the endowment effect is whether the effect survives in competitive trading settings. To test this hypothesis, a subject pool with varying degrees of experience as traders in real-world markets will be utilized. Neuroimaging techniques will be used to examine the neural responses of experienced versus inexperienced subjects. The experimental subjects will also be tracked over the course of several years to examine the persistence of behavioral patterns, and whether market experience attained during the survey period is reflected in neural activation patterns. In order to isolate the sources of social preferences/altruism displayed in laboratory games, an experimental design with variations in giving contexts will be used to test among competing explanations for the observed behavioral responses. Neuroimaging will play a crucial role in discriminating between alternative hypotheses and predicting behavior in novel situations. To collect neural measurements with high spatial and temporal accuracy, both fMRI and event-related electroencephalography (ER-EEG or ERP) techniques will be utilized, allowing a more complete understanding of the gains achieved from combining these two measurement modalities. The research is expected to have broader impact through several channels. The economic decision-making contexts studied in the research are ubiquitous in daily life. Assessing the effects of market experience and context variations on decision-making are necessary steps to extend the predictive domain of behavioral theories beyond the laboratory. The multidisciplinary nature of the project is expected to promote the flow of knowledge across the economic, psychology, and neurology/neuroscience disciplines, and will produce trainees (at the post-doc, graduate and undergraduate levels) who will further the communication between these fields.

Speaking is one of the most complicated human behaviors and yet speech is produced easily and with little conscious effort. Understanding the way the brain controls the various organs and muscles of vocalization is a basic scientific question that may illuminate more general principles that can describe how the brain programs and controls all behavior. In order to understand the neural mechanisms of motor behavior in vocal production, the proposed research will test specific hypotheses about the timing of different brain regions, the timing of muscles that produce vocal behavior, and the response to normal and abnormal auditory feedback. Understanding the neural mechanisms of motor control can have broad implications, including the development of more human-like robots, better computer-generated speech and vocal prostheses, as well as new therapies for treating articulatory disorders such as stuttering, dysarthria, aphasia, and other problems in speech production.

The proposed research will measure the timing of neural circuits that control vocalization in humans and song birds. The bird song production system has long been used as a model system that has relevance to understanding speech production. While simpler than humans and with less behavioral control of the experiments, studying birdsong is complementary in providing more granular measurements of neural activity in a vocal learning system than possible in humans. The experiments will test a novel hypothesis, that the motor system is not organized by the commonly assumed "top down" organizational scheme, but by a coherent network that operates as a unitary mechanism as described by certain mathematics of non-linear systems. The comparison between species will also be informative about how robust the results are across very different species and neural systems offering the possibility of generalization of the results.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Reaching for a pen to write or a tool to hammer a nail are the kinds of intentional actions produced every day. This project will advance our understanding the conscious control of behavior by investigating the neural mechanisms that underlie the sense of agency ? the feeling that one has both intended to and executed a particular movement. Scientists are now able to stimulate muscles electrically in order to enhance the performance of such routine movements. For some people, this externally stimulated action is felt to be the result of their own intentional control; for others, the action feels externally produced and not of their own agency. By measuring the distribution and timing of neural signals in these two situations of stimulated action that is felt to be intentional compared to movements that are felt to be externally controlled, the research will test cognitive and neural theories of consciousness and subjective experience in the initiation and control of behavior. The findings can lead to new ways of thinking about the neuroscience of action. They can also provide guidance for the improvement of user-system interfaces for robotic control and training methods for the future of work at the human-technology frontier, the development of new prosthetics as well as new rehabilitation therapies for the recovery of motor function following disease or trauma. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).

The research will enhance typically controlled human motor movements by manipulating them externally through functional electrical stimulation (FES), the use of a robotic exoskeleton, or the induction of illusory motion in order to probe the role of corollary discharge and reafferent signals in determining the time course and strength of the sense of agency. A set of six experiments will identify the role of agency in motor learning, the conditions under which agency is lost, and the extent to which voluntary movements can be externally modified without losing the sense of agency. Measurements of the activity and timing in the motor system and in somatosensory cortex using human electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) will be used to model neural responses. Measurements taken when enhanced movement is felt to be self-initiated will be compared to those taken when the movement is experienced as externally guided. This contrast between a sense of intentional agency in enhanced movement and passive unintentional movement will be related to the patterns of neural activity across different forms of external movement enhancement. The sense of agency for external stimulation of action has been predictive of improved motor learning and the neural signals related to agency will be used to model motor learning. These analyses will be used to test specific models of agency and consciousness in intentional action relating efferent motor cues, the striatal reward system, and learning. Understanding the neural mechanisms of conscious control of behavior can lead to new models for neuroengineering and brain-inspired design, provide new information about individual differences and variation in cognitive control of behavior and learning, and yield new understanding of how neural processes operate in realistic and complex environments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.