William W. Graves, Ph.D. - US grants

DESCRIPTION (provided by applicant): This research proposes to distinguish the neural correlates of phonological and semantic aspects of lexical (word-level) access. Studies of patients and functional neuroimaging studies of unimpaired subjects have implicated left-sided perisylvian (inferior frontal gyrus (IFG), posterior superior temporal gyrus (pSTG)) and extrasylvian (inferotemporal cortex (IT)) brain areas in lexical access. The specific aims of this application are: 1) To identify the neural systems involved in accessing lexical phonology; and 2) To identify the neural systems involved in accessing lexical semantics. Phonological word form (lexical phonology) access has been shown to be modulated by word frequency. With respect to aim 1, we hypothesize that producing lower frequency words will correlate with longer reaction times and greater fMRI activation in left superior IFG, pSTG, and regions of IT. With respect to aim 2, we hypothesize that processing more ambiguous words will correlate with longer reaction times and greater fMRI activation in the inferior region of IFG as well as more inferior areas of IT partially overlapping those associated with phonological access. This study should lead to a more complete neural account of lexical access.

[unreadable] DESCRIPTION (provided by applicant): The overall goal of this project is to supplement the applicant's existing training in functional magnetic resonance imaging (fMRI) with additional training in artificial neural network (ANN) modeling. In general, ANN models have been used to provide mechanistic accounts of behavior, while fMRI has been used to test for neural activation differences corresponding to differences in task conditions. To date there have been few efforts to combine the two approaches. Successfully doing so will enable an additional level of inference in which, on the one hand, a specific cognitive mechanism can be attributed to a particular brain activation pattern, and on the other hand, neural correlates can be ascribed to dynamic, quantitative models of behavior. Aim 1 is to develop methods for relating network activity and fMRI data. We hypothesize that successfully combining ANN models with fMRI will yield results similar to but not identical with results from more traditional task or stimulus based methods of fMRI analysis. The class of models considered here contains an input layer, an output layer, and at least one hidden layer, in addition to weighted connections between these layers. We will use an aggregate energy term to derive model-related activity associated with each stimulus presented to the model. When concatenated across stimuli, this model-derived signal will be treated as a prediction of the blood oxygen level dependent time course against which the signal in each brain image voxel will be tested. The expected advantage of the combined approach is that it should provide a clear functional interpretation of the fMRI results. Aim 2 is to apply these methods to the study of a few primary phenomena in reading aloud: word frequency effects, spelling-to-sound regularity, as well as sublexical orthographies and phonotactics. We hypothesize that network-derived regressors for these factors will yield more interpretive power than have standard regressors alone. The critical point is that interpretation of results from the combined approach is constrained by specific processing characteristics of the model from which the regressors were derived. Public Health Relevance: We aim to combine the complimentary strengths of fMRI and modeling to gain a deeper understanding of the brain, which will be invaluable in understanding and developing treatments for brain damage. This work could set the stage for a better understanding not only of language disorders, but, in principle, of any neurally-mediated disruption of behavior that can be modeled (e.g., amnesia, executive dysfunction, perceptual disorders, motor impairments). [unreadable] [unreadable] [unreadable]

The overall goal of this project is to better understand the brain basis of word recognition as it unfolds in both space and time. Accomplishing this will require fulfilling the training goal of supplementing the applicant's existing expertise in functional magnetic resonance imaging (fMRI) with new skills in magnetoencephalography (MEG). Word recognition is known to involve integration of orthographic (visual form), phonological (sound form), and semantic (meaning) information. What is not known is the degree to which this information is accessed sequentially or in parallel, the precise contribution of different brain areas to these distinct processes, or the exact order in which brain areas are engaged to support them. A detailed understand of this complex neural system is critical for understanding reading acquisition and developmental language disorders such as dyslexia, for which deficits in orthographic, phonological, and semantic processing can arise in various parts of the brain at various times. The methods used for investigating the neural basis of word recognition are generally optimized for measuring activity across either space (e.g., with fMRI) or time (e.g., with event-related potentials, ERP). Recently, MEG methods have advanced to the point where they can provide good spatial resolution, approaching but not equivalent to that of fMRI, while also yielding very high temporal resolution equivalent to that obtained with ERP. Studies integrating temporal and spatial results should, for example, help distinguish between automatic and controlled processing of orthography, phonology, and semantics. Starting from a neural model of word recognition with predictions about the spatial distribution of brain activation across time, we propose experiments to test the predictions according to three Specific Aims: (1) Specify the location and timecourse of semantic processing (mentored phase), (2) specify the location and timecourse of orthographic processing, and (3) specify the location and timecourse of phonological processing. Accomplishing these aims will lead to a more complete account of how, where, and when brain systems act to accomplish word recognition. The scientific knowledge gained is expected to be directly relevant to developmental and acquired reading disorders.

This Major Instrumentation Grant award supports the Acquisition of a GPU cluster to support interdisciplinary research in human learning, machine learning, and data science at Rutgers University--Newark, a Minority Serving Institution (MSI). It permits purchase of 3 Nvidia dual V100 GPUs to enable theoretical advances and practical applications in interdisciplinary understanding of learning. Rutgers-Newark is undertaking a multiyear effort to build strength in interdisciplinary computer science to support research training, and to address issues of diversity and representation within computer science and data science. These resources would: (1) enable the application of computationally-intensive methods in order to develop new theories and tools to understand human and machine learning; (2) support existing cross-disciplinary training efforts, such as graduate-level courses centered around deep learning and Deep Gaussian Processes; (3) enhance existing funded research by allowing the deployment of advanced data-analytic methods. The GPU cluster will provide a common computational resource for researchers from the Computer Science, Psychology, and Neuroscience departments through which they may collaborate to advance the state-of-the-art in each field. This purchase will complement the existing high-performance computing infrastructure already on campus as well as a recent NSF-supported purchase of a 1.2 petabyte storage system for cataloging the dynamics of human visual experience. Also, it will supplement an NSF-sponsored Mobile Maker Center for community-based data collection and fMRI research.

Humans remain the most powerful and impressive available models of learning, although the roots of these abilities are not fully understood. Although machine learning methods have become exceptionally powerful in recent years, they remain opaque in ways that human learning is not and still require vastly more data, energy and compute power than human learners. Both human and machine learning would benefit from the ability to more tightly connect and study the strengths of each. Gaussian processes provide one such unifying framework. They are an object of interest in machine learning, where they have dual interpretations as regression models and as neural networks, as well as in human learning where they have been proposed as models of cognition and perception. These multiple interpretations of Gaussian processes are key to their interest for bridging human and machine learning. From a theoretical perspective, Gaussian processes are equivalent to (a specific type of) neural network, but much more amenable to mathematical analysis, and can be stacked to obtain Deep Gaussian processes. This Deep learning framework may allow more systematic mathematical analysis than other Deep learning approaches---for example the ability to derive explanations for their inferences. The primary research goal of this project is to use the GPU cluster and the investigators' interdisciplinary expertise to draw deep connections between machine learning and human learning perspectives to advance the state of the art in both, while also improving data analytic capabilities.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.