Sayan Mukherjee - US grants

The inference problems associated with high-dimensional genomic data offer fundamental challenges for modern statistics, machine learning, and data-mining research. Methods that have had success in this domain impose constraints on models incorporating notions of simplicity, smoothness, or robustness. The constraints are often formalized either as priors for Bayesian methods or as geometric criteria for machine learning methods. The heart of this proposal is to develop and relate the importance of the geometry underlying the data to probabilistic modeling. The specific research foci of the proposal are: 1) The exploitation of geometric assumptions for problems of model uncertainty and variable selection in high-dimensional models; 2) A Bayesian framework for the use of ancillary or unlabeled data in predictive modeling; 3) Theory, methods and computation for nonparametric Bayesian kernel models; 4) Novel methods for nonlinear dimension reduction for high-dimensional data from regularization and geometric perspectives.

The proposal develops theory, methods and computational tools for statistical modeling motivated by applications in functional genomics. Modern molecular biology has generated data of a rapidly escalating scale and complexity -- high-throughput genomics data, genetic and sequence information, proteomic and metabolomic data, and other forms of more traditional biomedical or clinical information. Modeling this data for predictive phenotypes of prognosis, diagnosis, and pathway deregulation as well as understanding relevant variables and their associations are fundamental challenges for modern statistics, machine learning, and data-mining research. These methodological developments will have impact on several other scientific areas including biology, engineering, environmental and health science, and social sciences.

Data analysis is a fundamental problem in computational science, ubiquitous in a broad range of application fields, from computer graphics to geographics information system, from sensor networks to social networks, and from economics to biological science. Two complementary fields that have driven modern data analysis are computational geometry and statistical learning. The former focuses on detailed and precise models characterizing low-dimensional geometric phenomena. The latter focuses on robust or predictive inference of models given noisy high-dimensional data. This project aims to initiate a dialog between these two fields with geometry being the central theme. A closer interaction between them will benefit and advance both fields, and can potentially fundamentally change the way we view and perform data analysis.

Specifically, on one hand, the type of data common in the learning community poses several challenges for traditional computational geometry methods. The shift of focus to these challenges and the modeling of uncertainty central in statistical learning can broaden the scope of computational geometry, and lead to geometric algorithms and models that are more robust to noise and extend to high-dimensional data analysis. On the other hand, computational geometry has developed many elegant structures that contain often detailed and precise information about the underlying domain. Models parameterized using these structures can lead to statistical learning models and algorithms that are richer and more interpretable but remain robust to noise and are predictive.

This project is multi-disciplinary in nature, and will involve fields including computational geometry, algorithms, statistics, differential geometry and topology. Education will be integrated in this project.

In the past decade, the analysis of massive, high-dimensional, time-varying data sets has become a critical issue for a large number of scientists and engineers. Observations across several disciplines, by researchers studying dramatically different problems, suggest the existence of geometrical and topological structures in many data sets, and much current research is devoted to modeling and exploiting these structures to aid in prediction and information extraction. Recent work by the investigators, among others, has shown that integrating statistical methodologies with ideas derived from computational topology and diffusion geometry often leads to strikingly superior results than by conventional means. The investigators now propose to bring these methods into the mathematics/statistics curriculum and departmental structure in a formal way, by establishing a vertically integrated program of undergraduate and graduate research and education. This activity has broad support from programs within the Division of Mathematical Sciences, including Applied Mathematics, Computational Mathematics, Statistics and Topology programs, as well as Division of Mathematical Sciences Workforce Program.
This involves new undergraduate courses in the core theoretical areas, graduate topics courses, an extensive summer research program for undergraduates, as well as year long seminars aimed at both graduate and undergraduate students. In addition, the investigators will disseminate their ideas via summer workshops aimed at small-college faculty, and methodology workshops directed to faculty from other large research institutions.



The need to analyze massive, complex data arises in a wide variety of scientific areas of national importance, including for example satellite image analysis, medical genomics, and internet security.
The investigators propose a program of training future mathematical scientists to attack these new types of problems. The program will be vertically integrated, fostering extensive collaboration between post-docs, undergraduate and graduate students, and senior faculty, and will comprise a dynamic mixture of theoretical coursework and hands-on research activity. Participants in the program will gain valuable professional experience to distinguish them in the industrial and academic job-market.

The investigators have two aims in this proposal that fall at the interface of numerical algebra and statistical inference. The first aim is to extend the use of randomized approximation in a variety of dimension reduction methods that rely on numerical linear algebra both supervised and unsupervised as well as linear and nonlinear and develop a statistical bases for these methods in addition to the computational motivation of being applicable to massive data. The other motivation is to extend these statistical methods for dimension reduction to multiway data using numerical multilinear algebra, a recent new development in numerical analysis. These projects will increase interaction between statistical inference and numerical analysis and benefit both fields, providing new perspectives to how we view and perform data analysis.

Numerical methods with statistical implications are central to a variety of technologies used by the general population. These technologies include Google's pagerank algorithm, genetic methods used to find genetic variation related to disease, compressing of medical images for storage and treatment, as well as applications in geostatistics. In all the previous cases the fundamental idea is to condense massive data in a useful summary with respect to a desired goal. The two ideas in this proposal are (1) to study how numerical methods that scale to the massive data generated in modern scientific, engineering, and social applications impose statistical assumptions or models on the data, (2) to study more complex interactions or properties of the data than examined in current methods. The motivation behind the first aim is to understand how numerical approximations required for computational scaling as we collect more data impact the information that can be extracted from these data -- for what type of data and applications do certain numerical approximations work well. The motivation behind the second aim is to go beyond the broad category of standard statistical methods take into account the relation between pairs of objects -- two web pages that are linked for Google's pagerank, the correlation between two genes or two loci in genetics applications. The question behind this aim is whether richer sources of information can be extracted by examining the links between three web pages or three loci. The research involved in this aim consists of the development of computationally efficient algebraic methods to extract this information and understanding the statistical models implemented by these methods.

"Big data" have been growing in volume and diversity at an explosive rate, bringing enormous potential for transforming science and society. Driven by the desire to convert data into insights, analysis has become increasingly statistical, and there are more people than ever interested in analyzing big data. The rise of cloud computing in recent years offers a promising possibility for supporting big data analytics. However, it remains frustratingly difficult for many scientists and statisticians to use the cloud for any non-trivial statistical analysis of big data.

The first challenge is development---users need to code and think in low-level, platform-specific ways, and, in many cases, resort to extensive manual tuning to achieve acceptable performance. The second challenge is deployment---users are faced with a maddening array of choices, ranging from hardware provisioning (e.g., type and number of machines to request), software configuration (e.g., number of parallel execution slots per machine), to execution parameters and implementation alternatives.

This project aims to build Cumulon, an end-to-end solution for making statistical computing over big data easier and more efficient in the cloud. For development, users can think and code in a declarative fashion, without worrying about how to map data and computation onto specific hardware and software. For deployment, Cumulon presents users with best "plans" meeting their requirements, along with completion time and monetary cost to help them make decisions. A plan encodes choices of not only implementation alternatives and execution parameters, but also cluster resource and configuration parameters. This project develops effective cost modeling and efficient optimization techniques for the vast search space of possible plans. Cumulon addresses the challenges of uncertainty and extensibility (in terms of not only functionality but also optimizability). Cumulon also features a performance trace repository, which collects data from past deployments and uses them to improve cost modeling and optimization.

Cumulon aims to make statistical computing over big data easier and more cost-effective for a wide range of users including scientists and statisticians. Besides leveraging the cloud to provide on-demand, pay-as-you-go access to computing resources, Cumulon further simplifies development and deployment, reduces reliance on programming and tuning support, and accelerates data-driven discoveries. More than a one-shot solution, Cumulon is designed as a basis for an evolvable, open-source ecosystem that keeps up with advances in big-data analytics. Its repository of performance traces benefits the community in independent ways.

With the growing importance of quantitative, data-driven methods, Cumulon can impact many domains. The interdisciplinary team of PIs---from computer science, statistics, etc. ---is applying Cumulon to concrete applications in biomedical research and computational journalism. Through collaboration, the PIs seek to attract diverse talents, motivate them to work on problems with potential societal impacts, and help prepare them for the new challenges of big data.

For further information see the web site at: http://db.cs.duke.edu/projects/cumulon

Critical to advancing data-enabled science is our ability to probe, conceptualize, interpret, and visualize information residing in complex datasets in order to transform data into knowledge. This project develops computational methods and tools for investigation and visualization of structural variation in networks and 3D shapes, as well as inference of functional outcomes of such variation. These problems arise in areas of strategic interest such as health and medicine, where it is important to understand patterns of morphological variation in biological shapes and structural changes in biological networks, and their roles in behavior, health and disease. To emphasize this important interdisciplinary facet, the project is supported by several case studies that investigate: (i) mechanisms underlying the development of facial shape; (ii) interactions between brain shape, skull morphology, and behavior; (iii) organization of microbial communities in the digestive tract; and (iv) connections between social networks and microbiome networks. We envision many long-term ramifications of the project. Potential applications include analyses of dynamical social networks, exploratory discovery of associations between biological networks and phenotypic traits or diseases, quantitative studies of evolution, development, and inheritance of morphological traits, and challenges such as indexing and organizing databases of networks or 3D shapes for efficient data management, search, and retrieval.

The project integrates techniques from topological data analysis, integral geometry, spectral geometry, and statistics to develop computational methods and tools for modeling and visualizing structural variation in diverse collections of shapes and networks, and exploring associations between variation in structure and function. The project addresses theoretical foundations, computational methods, implementation of tools for statistical analysis and visualization, and validation of methodology. A shape or network is represented by a Borel probability measure on a Hilbert space whose elements represent Euler characteristic curves that encode rich geometric and topological properties. Dimension reduction and discretization of the probability measure lead to representations that allow organization of complex datasets into compact dictionaries that facilitate data analytics, processing, visualization, and search. This enables integration of the new methods with an array of existing techniques of multivariate statistical analysis to address such problems as development of regression-based models over networks.

This research will leverage ideas from algebraic and differential geometry to address core problems in modern high-dimensional and massive data science. The project will develop statistical methods and numerical tools, grounded in solid mathematical, statistical, and computational foundations, to extract low dimensional geometry from massive data with applications in clustering, data summarization, prediction, dimension reduction, and visualization. The solutions developed as part of this project can result in fundamental advances in practical applications across fields as diverse as biology, medicine, social sciences, communication networks, and engineering. In addition to internal validation via statistical and mathematical theory and simulation studies, the methods developed in the project will involve external validation via interdisciplinary applications. These applications include: (1) inference of population structure from genomic data; (2) document analysis via topic models; and (3) inference of subsets of putative gene networks relevant to drug resistance in melanoma.

The research is motivated by the central premise that, even though the amount of data may be massive, a compact model can represent these data. Specifically, high-dimensional and/or massive data can be reasonably approximated by a mixture of subspaces, for which sparse representations exist. A mixture of subspaces of potentially different dimensions is a flexible, rich representation of data with nice mathematical properties that can scale to large data. There are several fundamental challenges in modeling mixtures of subspaces that will be addressed in this research: 1) the subspaces will be of different dimensions, 2) both the subspace parameters and the mixing parameters need to be inferred, 3) efficient algorithms for inference are required for both high-dimensional and massive data. The central foundational impediment in all of these challenges is that the model is a stratified space (a union of manifolds), and therefore has singularities. The key insight in this research is that there exist embeddings and representations of the model space that mitigate these singularities. These ideas are implemented as concrete Bayesian, frequentist, and numerical algorithms and models to address the real world examples listed above.

Dynamical systems serve as important mathematical models for a wide variety of physical phenomena, arising in such areas as weather modeling, systems biology, and statistical physics. A dynamical system consists of a state space, in which a point represents a complete description of the state of the system, and a rule governing the evolution of the system from one state to another. This project focuses on the long-term behavior of such systems from two complementary points of view. From the first point of view, the project seeks to describe the behavior of typical systems when the rules of evolution are chosen at random. Such results shed light on what properties one might expect to find in disordered systems. The second point of view, the "inverse problem," concerns the statistical problem of recovering some information from the observation of a dynamical system. While there are many examples of dynamical systems being used as mathematical models, and there is a large statistical literature regarding inference and estimation, the performance of statistical procedures when applied to data generated by nonlinear dynamical systems is poorly understood. This project focuses on characterizing when traditional statistical procedures may be effectively applied in the context of dynamical systems. Beyond the very fertile potential applications, the project will also have broader impact on training of graduate students who will acquire invaluable skills in sound probabilistic modeling and statistical inference by working on the project's research topics.

Symbolic dynamical systems, which may be defined in terms of discrete constraints on the possible trajectories, serve as prototypical models of systems that evolve over time. Random ensembles of these systems may be produced by selecting the constraints at random. Ideas from both discrete probability and dynamical systems may then be used to analyze the structural properties of the resulting systems with high probability. For the inverse problem, this project seeks to evaluate the performance of several statistical inference procedures, both frequentist and Bayesian, when the models involved are dynamical systems. Fundamental questions about convergence and consistency of the procedures may be addressed using tools from ergodic theory, such as joinings and the thermodynamic formalism.

People and societies thrive best when they understand how the social and physical dynamics of their environment work, allowing them to respond appropriately. Natural scientists have built our understanding of the physical world. The scientific understanding they built has contributed to the development of technologies and practices that benefit human economies. For example, genetic sequencing of DNA enables deeper understanding of biological organisms; the consequences for human health, food production, understanding of evolutionary adaptation, etc. have been revolutionary and are still unfolding. The DNA sequence is the blueprint for an organism's anatomical structure (morphology) and function, but images capturing morphology are now much less prevalent than genetic data. Museums and researchers have been creating 3D digital images of natural history collections, and there are extensive 3D image data sets for some model organisms, but these data are mostly in closed collections, and generally unavailable or very difficult to access. This project aims to provide infrastructure to increase the accessibility of anatomical information, with a focus on 3D images. The resource will create the first open access, web-enabled image archive accepting and serving high-resolution, 3D scans of all organisms, called MorphoSource. Standardized descriptive tags will allow scientists to use this database to easily combine genetic and anatomical datasets for the first time, supporting the formulation of novel research questions. MorphoSource will link to other databases (such as iDigBio [www.idigbio.org]) that aggregate information on museum specimens from around the world. Having a shared common resource will change the culture among researchers and museums, making collaborations between physically distant experts more feasible, but it will also open the linked research collections of museums to anyone with Internet access anywhere in the world. Large data sets are prerequisites for many statistical and machine learning methods, so the resource will enable innovations in computational image analysis methods, fostering new types of collaborations that advance field-wide scientific understanding. The resource will track data use, enhancing reproducibility and also providing an objective metric of the value of individual data elements. Open access to the data linked through MorphoSource will enable anyone with Internet access to see the detailed anatomical evidence for theories like evolution. Pilot work has shown that teachers and students eagerly consume this newly available information, with numbers already in the thousands. Positive results of this access include (1) providing a more intuitive type of raw data (compared to DNA sequence) for showing the public why some conclusions about evolutionary relationships were reached, (2) providing an 'interest metric' for the value of natural history museums and the collections they hold, (3) increasing the community of people (including citizen scientists) who have access to the data required to make important discoveries by studying biological variation.

The specific plan for creating the repository for 3D data on all organisms is as follows. The primary goal is to restructure and improve a proof of concept database called MorphoSource. The restructuring will allow MorphoSource to meet the needs of a growing community of researchers and educators through massive upscaling, and to implement a novel approach for economically preserving data for the long term. To accomplish this, the MorphoSource server will be rebuilt to use the Fedora digital asset management architecture, which has been developed by library scientists to serve emerging needs related to the archiving and sharing of digital data. As part of this architecture upgrade, the data hosted on MorphoSource will be given an additional layer of protection through managing asynchronous copies in DuraCloud, a digital data preservation platform that leverages Amazon cloud. This restructuring will allow the MorphoSource server architecture to be integrated with the Duke University Libraries repository infrastructure. MorphoSource will also be able to invite institutional communities to be consortium partners in support of data storage and to enact data preservation techniques that guarantee integrity and readability for the foreseeable future. Additional tools will (1) allow for rapid, automated ingestion of dozens to hundreds of datasets at once, (2) link MorphoSource with major biodiversity archives, and (3) provide in-browser visualization of 3D series of image slices, such as those generated by CT and MRI scanners. The plan includes ingesting thousands of high quality legacy CT datasets from published studies, enabling their reuse, increasing the repeatability of studies. The project leaders plan to directly work with and design tools for K-12 educators and students to help them benefit from this resource. These datasets and educational tools will be available to researchers and the public through the updated MorphoSource website, available at www.morphosource.org.

A challenge in modern data science is how to integrate information from 3-dimensional shapes into statistical models. Examples of applications where this challenge is central is associating the shape of a tumor to molecular processes or associating the shape of the roots of a rice plant to crop yield. In graphics and anatomy, there is the related question of how to warp one object into another, such as the molar of a child to a molar of an adult. This project seeks to develop methodology to transform these shape data, such as meshes or 3-dimensional images, into representations for which standard statistical models are available. These methods are crucial to advancing data-enabled science in extracting, conceptualizing, interpreting, and visualizing information residing in datasets comprising complex objects such 3D shapes. Cutting edge ideas in mathematics, specifically from the field of geometry, will be used to address the fundamental problems of (i) modeling structural variation in diverse collections of shapes and (ii) modeling transformations between shapes. Solutions to these problems are crucial to many practical applications and disciplines including biology, medicine, social sciences, and ecology. As such, an important component of the project is validation of methods and tools through applications in radiology and anthropology

Geometric concepts beyond Riemannian geometry and smooth manifolds will be leveraged to develop novel statistical methodology to address complex challenges in data analysis. Methods and tools, grounded on solid mathematical foundations, will be developed for modeling complex objects (such as shapes and surfaces) and complex relations within data (such as multi-commodity flow or aligning shapes). The research will address two statistical challenges using geometric tools: (1) representing surfaces and shapes via integral geometry, and (2) learning group actions or transformations between pairs of objects using the geometry of fiber bundles. Addressing the first challenge results in a framework for parametric and non-parametric statistical models for collections of shapes and surfaces, without the requirement of landmark points and without requiring the shapes to be isomorphic. Addressing the second challenge provides a statistical framework for alignment problems ranging from aligning a collection of shapes such as teeth, to optimization problems on networks such as multi-commodity flow. The solutions proposed will impact statistics and geometry, and the methods may catalyze transformative advances in practical applications and scientific disciplines as diverse as biology, medicine, social sciences, and ecology.

Gut microbiomes are diverse microbial communities that play important roles in mammal physiology and health. However, scientists do not yet understand the rules that govern gut microbiome changes (such as how gut microbiomes change over time), or whether these rules are the same or different in different individuals. The goal of this project is to discover if gut microbiome dynamics are "universal" - that is, the same across hosts - or if these microbial changes are host-specific, such that each host's microbiome follows its own rules. Discovering whether each host's microbiome follows its own rules is relevant to human and animal health. If microbiome dynamics are universal, scientists can use these rules to design microbiome therapies that work the same way in different individuals. On the other hand, if these microbes are unique to each individual, then interventions to improve human or animal health must be designed separately for each individual. The work has benefits to broadening participation in STEM education through training in the investigators' labs and an educational outreach project with a public high school in Indiana. In addition, the researchers will make all statistical methods and code available online for use by other microbiome researchers and microbial ecologists.

Scientists are currently unable to test if gut microbiome dynamics are universal because they lack both the data and the statistical methods to analyze the data. This project overcomes these problems in two ways. First, it will develop new statistical theory and methods to jointly model multiple, highly multidimensional data sets such as those for microbiomes. Second, it will leverage an exceptional, longitudinal microbiome dataset, consisting of over 20,000 16S rRNA gene sequencing profiles on gut microbiome composition, generated for more than 500 wild baboons over a 15-year span. To these microbial composition data, this project will add data on the longitudinal, functional genetic dynamics of the gut microbiome via metagenomic sequencing. The baboons have been the subject of continuous, individual-based studies of baboon behavior and ecology for 47 years. As a result, this project has information on all of the covariates necessary to understand gut microbial dynamics, including the baboons' diets, abiotic environments, social interactions, and composition of the microbial communities themselves. The results of this project will be directly applicable to applied research on mammalian gut microbiomes aimed at improving gut microbiome function to improve host health.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This award supports TRIPODS@Duke Phase I, a project that will develop the foundations of data science both at Duke University and in the broader NC Research Triangle and surrounding region. A total of 25 faculty at Duke representing the disciplines of Computer Science, Electrical Engineering, Mathematics, and Statistical Science will be involved in Phase I. Activities include five semesters of workshops, with 3-4 one-week workshops each semester. These workshops will involve local and national participants and will bring experts on data science to the area. The project will support graduate students and postdoctoral trainees both in terms of education in the foundations of data science as well as in their professional development. Educational activities include the development and teaching of data science across curricula in Computer Science, Electrical and Computer Engineering, Mathematics, and Statistical Science, both at the undergraduate and graduate levels. The project will also leverage existing data science programs, including the Rhodes Information Initiative at Duke, a center for "big data" computational research and expanding opportunities for student engagement in data science; and the Statistical and Applied Mathematical Sciences Institute (SAMSI), one of the NSF/DMS-funded Mathematical Sciences Research Institutes (MSRIs), which is a partnership among Duke University, North Carolina State University (NCSU), and the University of North Carolina at Chapel Hill (UNC).

The topics of the signature workshops supported by the TRIPODS@Duke Phase I project are (1) scalable inference with uncertainty, (2) causal inference, (3) neural networks, (4) complex and dynamic image and signal processing, and (5) interpretable models. These five topics all fall under three research themes that require transdisciplinary collaborations among computer scientists, electrical engineers, mathematicians, and statisticians: Theme I: Scalable algorithms with uncertainty for data science; Theme II: Data science at the human-machine interface; and Theme III: Fundamental limits of data science. The potential research innovations for the three themes that will be developed and or advanced include: For Theme I, scalable Bayesian and generalized Bayesian inference, robust optimization for uncertain inputs, and algorithm and architecture design for neural networks; for Theme II, interpretable models and algorithms, causal inference with high-dimensional complex observational data, and image and signal processing for screening and monitoring; and for Theme III, robust optimization for uncertain inputs, statistical and approximation power of deep neural network architectures, and fundamental limits of causal inference in observational studies.

This project is part of the National Science Foundation's Harnessing the Data Revolution (HDR) Big Idea activity.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This project will address the problem of making inferences about sequences of observations that exhibit dependence arising from physical or other interactions. Observations of this sort occur in many fields, including finance, ecology, natural language processing, and biology. We will explore ways to fit a sequence of observations to a family of statistical models using ideas from the theory of optimal transport. Informally, we will identify models in the family into which the generating mechanism of the observations can be transformed with the least overall cost. We will address both the theory and efficient computation of these transformation costs, and will consider applications to biomedicine and computer science. The project will involve collaborations with graduate students and more senior researchers working in genomics and bioinformatics. Both undergraduate and graduate students will receive training through involvement in supported research projects.

On a more technical level, this project will address inference for stochastic processes, in particular, how to fit a family of stationary processes to an observed ergodic process, revealed sequentially. Research will focus on the use and extension of ideas from optimal transport, including stationary couplings of stationary processes and related variational quantities, with a focus on methods development and supporting theory. The research has two primary aims. The first aim is to investigate minimum divergence estimation based on joinings, including the use and properties of entropy regularization. The second aim is to investigate the efficient computation of optimal transition couplings of Markov chains, with applications to graph distances, graph alignment, and hidden Markov models. The project will involve collaborations with graduate students and more senior researchers working in genomics and bioinformatics. Both undergraduate and graduate students will receive training through involvement in supported research projects.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.