Dimitrios Pantazis, PhD - US grants

The brain-basis of perception is complex, and recent research suggests that neural processing depends on large-scale oscillation of neuronal units. Oscillatory cortical networks detected with electroencephalography and magnetoencephalography recordings often involve several frequency bands, indicating that a multivariate (multi-frequency) analytic approach would have better sensitivity in detecting neural effects than univariate analysis. However, popular connectivity measures, such as coherence and phase synchrony, typically analyze pairs of spatial locations and take into account a single quantity from each location, such as amplitude or phase within a specified frequency band. With funding from the National Science Foundation, Drs. Dimitrios Pantazis, Richard Leahy, and Jintao Jiang will develop robust multivariate statistical methods for detecting brain interactions in electroencephalography. Given the wealth of information in electroencephalography data, analysis using a single frequency approach requires either prior knowledge of the frequencies at which interactions occur or, conversely, a large number of tests, one for each possible type of interaction. In this project, the researchers are using canonical correlation analysis, which can find the optimal combinations of frequencies in one cortical site that best correlate with frequencies at another cortical site. In contrast to conventional methods of interaction analysis, this project is automating the identification of frequency bands that contribute significantly to cortical networks. The target application focuses on audiovisual speech integration effects. The multivariate methods developed in this proposal are being used to detect multisensory interaction cortical sites and account for different levels of phase-resetting from audiovisual speech stimuli with different stimulus onset asynchronies.

This research will facilitate the detection of oscillatory cortical networks both in the normal and pathologic brain. Changes in oscillatory brain activity have been reported in a wide array of neurological diseases, including epilepsy, schizophrenia, and Alzheimer's disease, and improved methodologies to detect the presence and differences in oscillatory activity and associated networks will in turn advance the understanding of these diseases and facilitate the development and assessment of therapeutic interventions. This effort brings together engineers and neuroscientists to tackle a broad range of scientific and technological problems, and as a result, the project offers opportunities for integrated interdisciplinary research training of doctoral students. Research results will be disseminated broadly to the research community through professional meetings and journals, and the developed research tools will be distributed to the research community through the open source software BrainStorm.

As Lewis Carroll famously wrote in Alice in Wonderland - It's a poor sort of memory that only works backwards-. On this side of the mirror, we cannot remember visual events before they happen; however, our work will help predict what people remember, as they see an image or an event. Our team of investigators in cognitive science, human neuroscience and computer vision bring the synergetic expertise to determine how visual memories are encoded in the human brain at milliseconds and millimeters-resolution. Cognitive-level algorithms of memory would be a game changer for society, ranging from accurate diagnostic tools to human-computer interfaces that will foresee the needs of humans and compensate when cognition fails.


The project capitalizes on the spatiotemporal dynamics of encoding memories while providing a computational framework for determining the representations formed from perception to memory along the scale of the whole human brain. A fundamental function of cognition is the encoding of information, a dynamic and complex process underlying much of our successful interaction with the external environment. Here, we propose to combine three technologies to predict what makes an image memorable or forgettable: neuro-imaging technologies recording where encoding happens in the human brain (spatial scale), when it happens (temporal scale), and what types of computation are performed at the different stages of storage (computational scale). Characterizing the spatiotemporal dynamics of visual memorability, and determining the type of computation and representation a successful memorability system performs is a crucial endeavor for both basic and applied sciences.