Liam Paninski - US grants

Project Summary The central problem in systems neuroscience is to understand the neural code, at both large and small physiological scales. Progress has been limited by a lack of sufficiently rich experimental data, a shortage of quantitative techniques to characterize and analyze the data, and an insufficient number of interdisciplinary researchers skilled in both systems neurophysiology and advanced statistical methods. Recent developments open new possibilities for collaborative efforts to tackle these basic problems. First, advances in multi-electrode recordings make it possible to study the simultaneous activity of large ensembles of neurons in a wide variety of experimental settings. Similarly, recent improvements in high-resolution voltage- and calcium-sensitive imaging technology now provide data capable of constraining highly-detailed biophysical models of information processing in single cells. A major bottleneck now is in analyzing and quantitatively understanding this data. Specific methodological advances in four fields are proposed: 1) encoding and decoding information in population spike trains; 2) single spike-train analysis and optimal stimulus design; 3) highly-detailed biophysical models and optimal processing of dendritic imaging data; and 4) information-theoretic analyses of sparse neural data. In each case, the investigator and his research group will develop novel mathematical models and tools for fitting these models directly to the observed data. Computer code implementing these novel techniques will be made publicly available to enhance the infrastructure for research and education. This work will have impact on the burgeoning field of neural prosthetics, which will require substantial improvements in our ability to design signaling interfaces between artificial and real neural tissue. Understanding encoding and decoding in populations of neurons and developing models that allow us to predict the effects of experimental perturbations to their behavior is key to this endeavor. This research on neural coding will also likely lead to mathematical results and statistical techniques which are of independent general interest and utility, with fundamental impacts on information theory, image processing, and optimal filtering and prediction of point processes (which in turn impact hundreds of other disciplines). In addition, the investigator is developing an advanced training course for graduate students and postdocs in statistical neuroscience (the first course of this kind in the world), as well as an introductory undergraduate course. Lecture notes will be made publicly available online and will shape a textbook in progress in advanced neural data analysis. Training opportunities will be pursued at Columbia University (strengthening already close ties with the Department of Statistics and Center for Theoretical Neuroscience) and with collaborators in the U.S. and internationally.

One of the greatest challenges in computational neuroscience is to reconstruct the connectivity of large, complex neuronal networks. The ability to decipher circuit connectivity would have a fundamental impact on our understanding of the dynamical properties and the functional organization of the nervous system. Knowledge of prevalent connectivity patterns will also shed light on the developmental constraints and learning rules under which the network might be operating.

Recent developments open new possibilities for collaborative efforts to tackle this basic problem. First, advances in two-photon imaging and photostimulation methods make it possible to observe the simultaneous activity of large ensembles of neurons, while stimulating neurons in arbitrary spatiotemporal patterns. Second, new statistical methods for extracting action potential timing information from calcium imaging data, and for modeling the response properties of small collections of neurons, are now efficient enough that they may be implemented on-line and scaled up to understand the function of large networks.

The investigators will combine these new experimental and analytical methods to estimate, for the first time, the connectivity diagram of large neocortical circuits, using two-photon calcium imaging of spontaneous and evoked activity in thalamocortical slices. A key novel step here is to directly verify the estimated circuit model with two-photon glutamate uncaging, which allows any neuron in the circuit to be activated (with single-cell resolution) while the evoked postsynaptic responses are monitored.

This interdisciplinary project has three complementary specific aims: (1) Develop statistically-optimal methods for real-time inference of spike timing from calcium imaging data. (2) Use these spike timing inference methods to estimate the network connectivity from large-scale multineuronal calcium-imaging of cortical slices. (3) Confirm the derived connectivity maps with glutamate uncaging and patch clamping, by photoactivating putative presynaptic neurons while recording intracellularly from postsynaptic cells. The proposed methods should also prove applicable to study other central and peripheral regions of the nervous system; data analysis software will be made publicly available online, to enhance the infrastructure for research and education in computational neuroscience.

Vision begins in the retina, where light is converted into electrical signals, processed to extract and compress visual information, and transmitted through the optic nerve to the brain. Despite decades of research, a full understanding of these transformations remains incomplete. In particular, most studies have documented specific properties of the responses of single retinal cells in isolation, using specialized artificial visual stimuli. The research performed under this grant aims to develop a full, unified computational model of retinal processing, including spatial and temporal filtering, nonlinear transformations, and adaptation to local luminance and contrast, in complete populations of neurons. The model will be tested by comparing its predictions to data from large-scale multi-electrode recordings of primate retinal ganglion cells (RGCs), verifying that it can mimic known retinal responses, and critically, testing its ability to explain responses to natural visual images, including the effects of fixational and saccadic eye movements. The resulting model will provide a compact encapsulation of the "neural code" of the retina, which will serve as a substrate for understanding all subsequent visual processing in the brain. In addition, the model will provide an essential component in the development of high-acuity retinal prostheses for people blinded by diseases of photoreceptor degeneration. Finally, the model will offer a useful tool for the development and testing of new display technologies.

The research has two main aims: (1) Develop and test a model of nonlinear subunits in RGC populations-- No current model captures the effects of nonlinear computations in a complete sensory neural circuit. The researchers will develop a model incorporating nonlinear subunits that captures the stimulus encoding properties of complete populations of RGCs at the resolution of photoreceptors, and will quantify the implications of these nonlinearities for encoding naturally-occurring visual stimuli. The researchers will develop methods to reliably fit the model to RGC responses to targeted stimuli that stringently constrain model structure, and verify model predictions in closed-loop experiments. (2) Incorporate adaptation; test model with targeted and naturalistic stimuli-- RGC responses adapt to luminance and stimulus contrast. No current model of the RGC population response incorporates adaptation with subunit nonlinearities, natural scenes, and eye movements. The researchers will incorporate adaptation in the model, fit the adaptive model using stochastic stimuli with varying mean and contrast, and test the model using stimuli that produce adaptation within and across subunits.

Understanding how the brain works is arguably one of the most significant scientific challenges of our time and the focus of the BRAIN initiative. It is widely believed that neural circuit function is emergent, the result of complex interactions between constituents with individual neurons forming synaptic connections with thousands of other neurons. Mapping of these complex circuits has been virtually impossible because of the reliance on electrophysiological recordings which sample these networks extremely sparsely. These tools for extracellular spike recordings are only able to simultaneously record from several tens to a few hundred neurons. Raw signals from these recording electrodes are first filtered to remove out-of-band signals. Putative spike events are then detected and extracted. Finally, these snippets of time-series event are sorted, typically on the basis of waveform shapes, into clusters. Even at the very modest bandwidths for these systems, computing systems struggle to save the data and process the resulting data sets. Scalability of these measurement techniques by many orders of magnitude in recording density and channels will be essential to future progress in understanding neuron circuits.

This project is exploiting emerging electrophysiological recording systems in which the electrode (and channel) count is increased by almost three orders of magnitude over conventional systems with data bandwidths exceeding 1GB/sec. To handle these data bandwidths and resulting data volumes and deliver scalability, this project will develop dedicated hardware and associated algorithms for spike detection and sorting that allow these tasks to be performed in real-time in close proximity to the recording system. Compression by more than three orders of magnitude is possible by these means by taking advantage of the special spatiotemporal local structure in these data sets; by exploiting strong prior information about the spiking signal and reducing the dimensionality of the problem accordingly; and by adapting and extending modern scalable nonparametric Bayesian inference methods. In addition to providing important new tools for neuroscience, the tools developed here for scalable real-time event detection and annotation have broad applicability to other spatiotemporal data sets (or more generally, any data set comprising multiple streams of data, in which the streams could involve different data modalities) in which objects of interest are spatially and temporally localized with fixed spatial footprints. Examples abound in cell and molecular biology, particle and solid-state physics, financial monitoring, monitoring of power networks, and sensor networks.

Optimal calcium imaging with shaped excitation Understanding information flow in the brain is dependent on simultaneously recording the activity of large neuronal populations. It seems impossible to interrogate neurons serially, and still image large populations of neurons with high temporal resolution and high signal to noise. This is linked to the inverse relationship between volume scanned, and the signal collected per voxel, at fixed spatial and temporal resolution. However, this is not a hard limit. The goal of most functional imaging is to recover and assign activity signals from neurons; here we demonstrate that nearly all past approaches have dramatically oversampled spatially to create human-interpretable images. This is not necessary ? once the spatial footprints of the observed neurons are known, constrained non-negative matrix factorization methods can extract highly accurate temporal activity signals from very low spatial resolution movies. Reducing the number of samples required in imaging allows us to significantly speed up acquisition. In this proposal we introduce a new fast computational imaging method, leveraging modern computational demixing methods with simple optical hardware to increase imaging speeds by an order of magnitude. We proposed to use modern spatial light modulator systems to provide a flexible and powerful tool for optically implementing our proposed spatial downsampling approach, while taking better advantage of laser power and avoiding standard problems with diffraction- limited imaging caused by limited dwell times on the sample. The resulting combined hardware- software solution will be inexpensive, easy to implement and maintain, and widely applicable in the hundreds of labs currently using multi-photon imaging methods. Thus, the proposed approach will enable a critical leap towards achieving the goals of the BRAIN initiative.

Understanding how a healthy brain interprets sensory signals and guides actions, and why an unhealthy brain fails to perform these functions properly, is a profound and ambitious goal of 21st century science. Integrating knowledge of neural circuit function into a coherent picture of perception, cognition and action requires extraordinary cooperation and coordination between three research areas: experimentation, data analysis and modeling. The National Science Foundation Theory Team at Columbia University will unite exceptional resources in statistical data analysis and theoretical modeling with an extensive network of experimental collaborators to address the enormous challenges facing neuroscience. Never has the need been greater for theoretical insights and sophisticated data analysis. The field of neuroscience is facing a torrent of complex data from a system that is, itself, extraordinarily complex. Future progress requires developing the ability to extract knowledge and understanding from these data through analyses and modeling that capture the essence of what they mean. The goal of the NeuroNex Theory Team at Columbia is to establish, through the quality of its research, the excellence of its trainees, and the impact of its visitor, dissemination, and outreach programs, a new cooperative paradigm that will move neuroscience to unprecedented levels of discovery and understanding.

High-density electrode recording, wide-field calcium imaging and complex connectivity mapping are bringing neuroscience into an era of extensive multi-area and even whole-brain studies of neural activity and circuitry. The neuroscience community desperately needs new ways of interpreting data obtained from different species using myriad techniques and for thinking about neural processing over large length and time scales and across multiple brain areas. In response to these challenges, two major goals will drive and define research at the NeuroNex Theory Team at Columbia: first, integrating the analysis methods and theoretical models used to infer meaning from data with each other and with the experiments that generate these data; and second, providing analytic tools and theoretical frameworks to understand interactions between multiple brain regions and to draw important overarching lessons from experiments exploiting a variety of techniques across different species. Progress will be made through a tight integration of theoretical techniques with outstanding experimental collaborators working on a variety of systems and species. Graduate and postdoctoral training will stress technical excellence and broad perspectives in both theoretical and experimental neuroscience. Outreach will be made to other researchers through visitor and exchange programs, sponsored meetings and dissemination of research results and high-quality, user-friendly software. Outreach will be made to the broader community by sharing the excitement of neuroscience research with elementary and high school students and with the general public. This NeuroNex Theory Team award is co-funded by the Division of Emerging Frontiers within the Directorate for Biological Sciences, the Division of Physics and the Division of Mathematics within the Directorate of Mathematical and Physical Sciences, and by the Division of Brain and Cognitive Sciences within the Directorate of Social, Behavioral and Economic Sciences, as part of the BRAIN Initiative and NSF's Understanding the Brain activities.

SUMMARY The major theme of this proposal is a tightly closed loop of experiment, theory, and data analysis. Sophisticated, scalable data science methods are a critical component of this loop. The Data Science Core serves two primary purposes. First, we will apply and refine sophisticated data analysis algorithms directly related to the project?s scientific goals. This project will generate massive streams of data from multiple recording and simulation modalities: whole-cell electrophysiology and anatomy, large-scale calcium imaging, spatiotemporally-complex optogenetic perturbations, RNA sequencing images, in addition to massive simulations of networks of spiking neurons. A correspondingly major effort is needed to manage this data, to distill it into new scientific knowledge, and to design new experiments, theoretical analyses, and simulations to close the theory-experiment-analysis loop. This will entail the application and iterative refinement of algorithms for preprocessing the data (e.g., taking calcium imaging video and extracting demixed and denoised neural activity from each cell visible in the field of view); aligning, registering, and performing statistical inferences on data across multiple modalities (e.g, calcium imaging, optogenetic stimulation, and seqFISH); functionally characterizing the stimulus preferences and correlation structure of the activity in the observed cells; and developing closed-loop optimal experimental design methods to obtain richer, more informative data. Second, this Core will build a collaborative infrastructure allowing the multiple laboratories in this project to act as one: sharing data and analysis tools, and closely integrating theorists and experimentalists. This infrastructure will: be completely open source; build on current efforts to standardize neuroscience data; be modular and extensible to allow for rapid iterative improvement of each stage of the algorithmic pipeline; enforce automatic archiving and recording of algorithmic metadata describing versioning and parameter choices for easy searchability and reproducibility; and allow for straightforward benchmarking. As we develop these practices and tools for data and analysis pipeline sharing, we will make them immediately available to the community. Thus we will provide a model platform for vastly improving reproducibility, keeping analysis pipelines up to date as improved methods are developed, and most importantly saving researchers from re- developing and re-implementing analysis software and data storage/sharing solutions. We aim to make it easy for groups of labs anywhere in the world to unite and crack large-scale neural circuits. This will transform the way neuroscience is done.

The development of the nervous system, specifically the dynamics of neuronal development and wiring to build brain architecture and their constructive role in emergent brain activity, constitutes a central unexplained phenomenon in living systems. The study of developing brains requires a comprehensive and systematic characterization of the brain of an organism at different ages and a suitable mathematical framework, able to capture the structure of the growing nervous system and the emerging networks therein. We propose to address this fundamental challenge by developing such a mathematical framework capable of characterizing underlying network changes in living brains and their consequences for functional neural activity and resulting behavior. This mathematical framework will be applied to analyze the complete nervous system, at single-cell precision, of the model organism C. elegans. To address these important challenges, we have assembled an interdisciplinary team with expertise in topology, computational biology, statistics, theoretical physics, neuroscience and biology of the model organism. Our group will develop new mathematical, statistical, and computational tools to characterize the structure of developing brain networks. This analysis will reveal shared-organizational, emergent principles of nervous-system development and function. Based on the widespread representation of biological data as complex networks and the universality of the mathematical, statistical, and computational methods we will develop, we expect wide applicability beyond the original system.

The aforementioned approach will be led by experiments that aim at providing multiple views of a developing network and their functional consequences to whole-brain activity. We will analyze the brain at two levels: changes to the underlying network as a consequence of extensive neural additions and connective neural (re-)wiring. We will compare the developing network at two transition periods: early maturation from the first to the second larval stage and, later, maturation of the two different sexes. In both of these developmental periods, newborn neurons grow the existing brain network, considerably, by roughly a third in size. In order to characterize the global properties of the data collected from these two different layers (neural network and brain activity) and to study the maps between them, we will develop tools based on topological data analysis (TDA) and Bayesian inference techniques. TDA provides methodology derived from algebraic topology that can be used to extract global features in large datasets. As a relatively new field, there are several major roadblocks that obstruct the wide applicability of TDA to biological systems, including the development of statistical approaches, comparison (homomorphisms) of networks (simplicial complexes), and time-series analysis. These tools will be then applied to study biological datasets that describe the developing brain network and changes to neurobehavioral activity therein. In particular, we will characterize basal networks and those for attractive and aversive behavior, for whole brains at a single-cell level, during developmental transitions that are known to restructure this behavioral network at both the level of input and output.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.