Guillermo Sapiro - US grants

IBN-9720305 PI: SHAPLEY This project is being funded through the Learning & Intelligent Systems Initiative. The questions involve vision and learning in the central nervous system. The project investigates how the brain represents and processes perceptual information, and the adaptive changes that occur when the system learns and improves its performance. Here the visual system is used as a gateway into the workings of the brain, employing methods from psychophysics, neurophysiology, mathematics and engineering. The work focuses on visual grouping, which is the ability of the visual system to link together local elements of the visual image into coherent wholes. Grouping is one of the most fundamental aspects of human vision, and the goal of this work is to obtain a theory of visual grouping that will explain the physiological and psychophysical data, and lead to new technological ideas to be applied in intelligent artificial systems. Results will be important because understanding the computations in the brain that produce grouping would be a leap forward in our understanding of brain function, and of any systems that can adapt to experience. This work will therefore have an impact in designing learning materials and in the optimal methods of presenting information, and in the design of the next generation of computer vision systems and intelligent control systems. This project is supported in part by the NSF Office of Multidisciplinary Activities in the Directorate for Mathematical & Physical Sciences.

Sapiro

This project addresses the development of a comprehensive research and educational program in mathematical techniques applied to the areas of image processing and computer vision. In particular, Geometric Partial Differential Equations (GPDE's) are used. Using GPDE's is a relatively new approach to image analysis that brings numerous benefits and has already lead to several state-of-the-art results. This research focuses on the theoretical and practical study of ways to introduce advanced a-priori and learned knowledge and information into the GPDE's framework, moving beyond the simple edge information commonly used.

Key examples where knowledge can be incorporated include MRI and SAR, where the number of different objects present in the image is known a-priori; object tracking in video, for human-computer interaction for example, where information from previous frames can be used; and medical image analysis and visualization, where important biological information is available. This knowledge is incorporated combining Bayes rule, learning techniques, systems of coupled GPDE's, and special geometric forces driving the GPDE's. The theoretical study of these new equations is an integral part of the program. The educational component of this project focuses on improving multi-disciplinary training and creating a new image processing laboratory based on personal computers. New courses on GPDE's and on Geometric Visual Tracking are being introduced. These and the basic image processing courses are oriented to a diverse audience and are attended by students from many departments across the University. The collaboration with industry and other departments is also addressed defining joint projects. The whole research and educational program is oriented toward making advanced tools in image analysis more accessible to users that frequently interact with visual information in their personal computers.

This project seeks to develop a comprehensive research and education program in the area of computational
methods and simulations of physical systems described by high order Partial Differential Equations (PDEs). The program will unify algorithmic, visualization, theoretical, and experimental efforts as well as address applications in areas of science and technology, including computer graphics, image processing, biology, and fluids.
Intellectual merit of the proposed activity
This project advances knowledge in the area of high order PDEs, with particular emphasis on curved surface data, and produces enabling technology to address fundamental problems in biology, image processing, computer graphics, and fluids in general. The novel science is in the computational
techniques, experimental research, and diverse applications addressed by a multi-disciplinary team. This
project brings together the five fields of computer science, applied mathematics, mechanical engineering, physics, and electrical and computer engineering.
Broader impacts of the proposed activity
With the increasing interest in high order PDEs, the computational tools and experience resulting from this project impact beyond the particular applications in this proposal. Students will receive unusually broad
interdisciplinary training and the workshop planned further brings experts from different fields together. New public domain software incorporating the developed algorithms enables researchers from different fields using higher order PDEs to perform state-of-the-art numerical simulations and graphics rendering of their application of interest.
Educational initiatives of this research program include: (1) new interdisciplinary training of graduate students and postdocs through co-mentoring by PIs in different fields; (2) new interdisciplinary courses in computer graphics, numerical analysis, and modeling/simulation of physical phenomena described by higher order PDEs; (3) a workshop bringing together for the first time diverse scientific researchers using high order PDEs.

Almost all disciplines in information sciences and engineering use and need to compute the distance between events and geodesics or optimal paths between them. Geodesics are used for example for path
planning in robotics, brain analysis in computational neuroscience, image segmentation, and finding fundamental geometric objects on surfaces. In high dimensions, they are the building blocks for problems in data mining and information discovery, with applications ranging from finance to biology to image and video processing. In general spaces, generalized geodesics also have numerous applications, from brain imaging to color image enhancement to computer graphics

Since distance functions and geodesics are fundamental for the interpretation and utilization of
data and the knowledge and information embedded in it, it is important to have a good theoretical understanding of generalized distance functions and geodesics, and to develop efficient techniques for their accurate and fast computation. The work is needed in high dimensions and for diverse types of data representations, including implicit hyper-surfaces, sets of unorganized points such as manifold samples obtained when learning high dimensional information from examples, and point clouds from popular 3D range scanners. This project addresses these theoretical and computational issues, and concentrates on specific important applications in the areas mentioned above. These applications can be efficiently approached only once the theoretical and computational frameworks here studied have been derived.

In the educational arena, the goal is to continue with current efforts in the interdisciplinary
aspects of advanced education and in the distribution and popularization of the developed techniques,
including making software publicly available.

0404617
Sapiro

This U.S.-France cooperative research project between Guillermo Sapiro's research group at the University of Minnesota and the Odysee research laboratory led by Olivier Faugeras and Rachid Deriche at the French National Institute for Research in Informatics and Applied Mathematics (INRIA) in Sophia Antipolis focuses on the development of computational tools for brain imaging studies. The research addresses brain development and shape deformations, diffusion tensor imaging analysis and analysis of functional brain data. They will apply advanced computational tools based on partial differential equations to data obtained from the Center for Magnetic Resonance Research at the University of Minnesota.

Intellectual Merit: The project will advance knowledge in brain imaging research, using and developing novel computational tools. The research addresses theoretical and computational frameworks in an interdisciplinary research area.

Broader Impacts: The proposed computational techniques have applications in brain imaging and in the general area of medical imaging techniques. The collaboration takes advantage of French expertise in computer and biological vision perception and functional imaging and modeling of brain activity. Through this award, U.S. students are given the opportunity to develop research skills in an international research environment and partnerships with French researchers for future collaborations.

This project addresses the area of image and video inpainting, the technique of modifying an image in an undetectable form. The goals and applications of inpainting, which is as ancient as art itself, are numerous, from the restoration of damaged paintings and photographs to the removal of selected objects in a scene and the discovery of data behind occlusions. Image inpainting is also important for less-conventional applications such as wireless transmission and compression and image integration from camera arrays. The goal of this project is to develop a working solution to image, video, three dimensional, and multiple views inpainting and to address applications such as image and film restoration and alteration, image recovery from camera arrays, and filling-in of holes in 3D range data. Biological aspects of inpainting, through its fascinating possible connections with camouflage and the filling-in of the human blind spot, are investigated as well. The theory and applications of this project cover very diverse areas such as image processing, partial differential equations and fluid dynamics, numerical analysis, art, computer graphics, entertainment, and biology.

The inpainting problem is addressed as the computation of a smooth continuation of the available spatial and
temporal information surrounding the region to be filled-in. This is achieved via partial differential equations, which incorporate principles ranging from fluid dynamics to Gestalt's smooth boundary continuation. The combination of these techniques with more classical texture synthesis algorithms is also addressed and exploited in this project. The research addresses the problem in its generality, assuming diverse and large regions to be inpainted, and derives both fundamental theoretical aspects and practical
working systems.

The study of earth's topography has fundamental impacts on society, from flood and landslide prevention and control to the understanding of climate change impacts, management of land-use practices, as well as design of roads and other man-made projects in an environmentally sustainable way. The recent availability of high resolution (0.5 m spacing) digital topography from airborne laser swath mapping and ground-based lidar offers opportunities to develop a new class of environmental predictive models that explicitly incorporate important features of the landscape and thus enhance the accuracy of predictions. The goal of this project is to develop modern computational geometric image analysis methodologies applicable to hydrologic and eco-geomorphologic hazard prediction and control. Specifically, the project studies high-resolution, multiscale, and dynamic topography with the goal of extracting channel networks, channel banks and shapes, floodplains and hazard-relevant features such as landslide prone areas and service roads which contribute to increased sediment production and thus stream habitat deterioration.
The mathematical and computational techniques to be exploited and developed come from the area of geometric non-linear partial differential equations and energy formulations, combined with differential and computational geometry. Specifically, a combination of methodologies ranging from geometric scale-space theory to singularity theory and geometric variational principles, combined with optimal algorithms for computing special curves on surfaces, will be exploited to derive a complete and automatic analysis of the topography at multiple relevant scales.

Efficient signal representation is critical in virtually all
disciplines of science. Sparse representations have recently drawn much
attention from the signal processing community. The basic model
consists of considering that natural signals admit a sparse
decomposition as a combination of very few atoms in some redundant
dictionary. Recent results have shown that learning overcomplete
non-parametric dictionaries for image representation, instead of using
classical off-the-shelf ones, significantly improves numerous image and
video processing tasks. This research aims at developing a
comprehensive theoretical, computational, and practical framework for
learning sparse representations for numerous signal analysis tasks.

First, the research concentrates on learning sparse representations for
global and local robust image classification tasks, proposing
formulations with both sparse reconstruction and class discrimination
components, jointly optimized during dictionary learning. Multiscale
dictionary learning and investigating a number of critical optimization
challenges are integral components of this project as well. The
framework of learning multiscale sparse representations is then
extended to multimodal data. As in the work with images and video, the
energies proposed for multimodality consider both reconstructive and
discriminative terms, learning the optimal representations both for the
given data and the given tasks. In addition to image, video, and audio,
other signal modalities studied include tensors, which are critical in
diffusion MRI, and bring the additional challenge of developing sparse
representations for non-flat data. The proposed work is also extended
to learning to sense, where combined with the learning of the optimal
dictionaries, the learning of the best linear sensing procedures is
studied.

This subproject is one of many research subprojects utilizing the resources provided by a Center grant funded by NIH/NCRR. The subproject and investigator (PI) may have received primary funding from another NIH source, and thus could be represented in other CRISP entries. The institution listed is for the Center, which is not necessarily the institution for the investigator. Using a powerful magnet, this study is aims to obtain extremely high-resolution images of cortical vessels, generate a 3-D model of the vascular tree and correlate it with the fMRI signals. The outcome of these studies will greatly enhance our understanding of the vascular network and benefit a variety of research applications including fMRI, cerebrovascular disease, and cancer angiogenesis.

This project will develop algorithms to assist with the early diagnosis of children who are at risk of developing behavioral disorders. Previous research has indicated that two critical areas of behavioral investigation for use in identifying at-risk children have been abnormalities in motor activities and emotional range displays, especially of the face. Motor abnormalities are based on the observation that motor control involves the circuits of the brain associated with dopamine; these are also implicated in behavioral disorders. Many different disorders share the observation of disruption in the emotional range regulation, so facial expressions are included in the study.

To date, assessments of motor and emotional range have been done by the experts who view and rate videos of an individual. However, these expert, subjective ratings limit the analysis of behavioral conditions to only a narrow range of behaviors, work only for small populations of individual subjects, and are both costly and dependent on the observer's particular expertise. In order to enable wider population screening, automation is required. Innovative ways of capturing and quantifying the expertise of experts will be accompanied by metrics for assessing the evolution of the behavior. In addition, new computational tools will support evaluation of the effectiveness of interventions.

The broader impacts of the proposed work will involve improved mental health levels across the populations by providing a systematic approach for enhancing early detection, prevention, or mitigation of behavioral disorders and likely reduce the long-term costs of missed or late diagnosis. The research results will be blended with the educational process through inclusion of project themes in the curricula at the Institute of Technology, the Medical School, and the College of Education and Human Development at the University of Minnesota and the creation of a program with annual workshops, tutorials, web pages and a wiki on knowledge discovery and behavioral analysis. The team will develop an interactive exhibit for children at the Bakken Museum, and create of new instructional material for student teachers at the Institute of Child Development and similar institutions. Development of a central web repository will insure that the algorithms and the data will be readily available for appropriate research.

Abstract
Proposal #: 10-39741
PI(s): Papanikolopoulos, Nikolaos; Lim, Kelvin; Guillermo Sapiro
Institution: University of Minnesota
Title: MRI/Dev.: Video-Based Robotic Instrument for Behavioral Analysis and Diagnosis of At-Risk Children

Project Proposed:
The proposed set of tools constitutes a video-based robotic instrument which targets the domain of early diagnosis for children at risk of developing psychiatric disorders. As such, this proposal is at the disciplinary boundaries between computer science, psychology and psychiatry, and medicine. Proposed is the development of a robotic instrument that could observe and automatically analyze abnormalities in children, thus introducing a novel technology which can help identifying children at risk. Specific activities include:
- Development and clinical verification of instrumentation and clinical protocols to quantify mental disorders in children;
- Development and usage of computer vision and machine learning methodologies in the instrument;
- Development of statistical models to evaluate the available related data sets;
- Usage of a wide array of passive and active sensors and state-of-the-art 3D camera systems to collect and analyze the monitored data;
- Usage of robots and robot pets as a means to detect and treat mental disorders; and,
- Practical validation of the instrument at the Medical School.

Broader Impacts:
The recent usage of computer vision methodologies/hardware and robotics for detection of mental disorders in children, in itself, constitutes strong broader impacts. Planned are also educational programs (workshops, tutorials, etc.) that will enable training gathering of physicians and psychologists to the aforementioned methods/procedures, which would otherwise not be possible. Moreover, significant planned curriculum development at the participating institutions revolves around the instrument. In addition, outreach activities for middle-school students from underrepresented groups will take place, and so will outreach to various pertinent patient groups. This truly interdisciplinary project also plans to include international partners.

This project develops mathematical and computational approaches for big data exploitation. Fast and online
algorithms that learn and adapt as data arrives and changes are developed. How to automatically
understand and reduce redundancy in the data, for a given task, is also addressed in this project. Big data
comes in multiple forms, e.g., audio and video, audio and text, video and weather, video from multiple
sources, brain imaging from multiple modalities, friendship networks and individual preferences. This is
also addressed in this project. The broad impact of the research is born in the large and diverse
applicability of big data and in the techniques here developed. In the education arena, the developed
Internet classes have an audience of tens of thousands, and the project provides unique integration of
research and undergraduate education via different Duke initiatives.

The framework follows the parsimony theory of sparse modeling. Challenges are addressed with a gamechanging
paradigm: learning to optimize; on-line learning what the task-dependent optimizer is expected
to do, developing computationally efficient algorithms to approximate the ideal behavior of sometimes
unknown optimizers. The work derives novel multi-modal formulations for network inference, and realtime
on-line robust PCA and robust NMF, fundamental tools in big data modeling and exploitation; as
well as robust 3D shape, networks, and multi-modal matching. The formulation elegantly solves bilevel
optimization problems rendering it efficient for classification and signal separation tasks. Sparse
modeling is extended to new venues and algorithms, making such techniques usable for big data. The
formulations and theoretical foundations are complemented with numerous applications.

Drone-based threat detection enables unprecedented coverage and flexibility in understanding human dynamics, with applications to real-time identification of unusual events and forecast of future threats. With these new possibilities come unique challenges, from highly dynamic scene changes to the need for low-cost operation. This project focuses on the foundations of video analysis technology for such dynamic drone-based threat detection. The work ranges from mathematical foundations in the area of learning and modeling to applications such as people tracking and identification. In terms of data, the project includes collection and analysis of drone-based video data, sharing data and the developed code with the community at large. The project will not only contribute to the emerging area of drone-based threat analysis but will also provide fundamental building blocks for modern visual data exploitation. Components of this project will be incorporated in online image-processing classes.

The work investigates fundamental problems motivated by drone-based video analysis, including orientation invariance, image hashing, multi-modality modeling, and progressive unsupervised self-learning. The project develops and exploits underlying mathematical foundations, such as subspace modeling and invariant filter design. All the work has efficiency as its goal; this being manifested from the development of memory and computationally efficient forest hashing to the development of oriented response networks with significantly reduced deep models for orientation invariance. To enable state-of-the-art performance, the project utilizes successful machine learning frameworks, including deep convolution neural networks, random forests, hashing, and latent-SVM. This is approached with fundamental enabling redesigns and developments in the areas of robust learning, invariant learning, unsupervised self-learning, and multimodal hashing. The contributions are critical for data-limited learning, cross-modality learning, and computationally/memory efficient systems. The project aims to develop and exploit underlying mathematical foundations, such as subspace modeling, invariant filter design and learning, robust geometry-based learning, and information-based code aggregation. The theoretical and computational contributions are expected to result in efficient implementations of threat detection for dynamic environments, drone videos being a particularly important example.

Data comes in all forms, including visual, text, medical records, and comments on social medial. This diverse (multimodal) data is critical when data is scarce, noisy, and uncertain. Different modalities can improve joint inference and decision making and allow producing (at extremely low-cost) results that were possible only with high-end devices and techniques before. In addition, inferring a condition from unexpected data sources is of paramount importance in disciplines ranging from marketing to health-care and defense. This project addresses these fundamental challenges with new mathematical and computational tools. Through new collaborations, the project has access to unique data and problems of significant impact in human well-being. A related online class also continues to grow and develop, with over 120,000 students so far. A unique summer immersion program will also involve undergraduate students in multimodal data science research.

The exploitation of multimodal data is one of the unifying themes of this project. A further unifying theme is the underlying mathematical foundation: subspace modeling and embedding. Tools from subspace modeling in the form of learning multimodal low-rank representations, modeling multimodal sparse networks, and solving for big data matrix decompositions are here developed. A third unifying motif of this work is the ubiquitous consideration of computational efficiency. All the above is developed in three major components: classification and recognition, data augmentation, and network analysis. The project addresses critical problems such as multimodal face recognition, dynamic multimodal graph inference, gaze analysis, multimodal network analysis, and non-negative matrix factorization. The overall goal is to efficiently exploit and integrate multimodal data to help in joint inference and decision making.

Cervical cancer is the second leading cause of death for women worldwide; 85% of deaths occur in low and middle- income countries (LMICs), despite the fact that well-established interventions exist for pre-invasive disease. In the U.S., screening for cervical cancer is performed with the Papanicolaou (Pap) smear. Colposcopy, which visualizes the acetic acid stained cervix with a low power microscope, followed by biopsy of cervical abnormalities, serves as a confirmatory test for women with positive screening results. Women with pre-cancer are treated via excision of a portion of the cervix using Loop Electrosurgical Excision Procedure (LEEP). Women with cancer are referred to a combination of local and/or systemic therapy depending on the stage of invasive disease. This model is not practical to implement in medically-underserved regions due to lack of resources to procure, implement, and maintain the technologies in the care cascade. Thus, alternative protocols that employ low-cost, simple-to-use technologies are needed to mitigate cervical cancer. Our vision is to develop high quality, low-cost interventions that will be effective in low-resource health facilities to address shortcomings of current technological solutions to cervical cancer prevention. This vision requires 3 distinct innovations: 1) an ultra-portable visualization device coupled with routinely used contrast agents to reduce discomfort, enable visualization of microscopic disease, and provide quality control through image review and archiving (Aim 1); 2) enhanced contrast and smart algorithms to reduce unnecessary referrals or treatment of screen positive women (in the absence of confirmatory biopsy) (Aim 2); and 3) a low-cost therapeutic that is as effective as current ablative approaches but more readily accessible to treat pre-invasive disease (Aim 3). These innovations build upon: 1) a novel Pocket colposcope developed under previous funding that enables portable colposcopy of the cervix; and 2) preliminary results that support these new innovations.

SCALABLE COMPUTATIONAL PLATFORM FOR ACTIVE CLOSED-LOOP BEHAVIORAL CODING IN AUTISM SPECTRUM DISORDER ABSTRACT Despite significant recent advances in molecular genetics and neuroscience, behavioral ratings based on clinical observations are still the gold standard for screening, diagnosing, and assessing outcomes in neurodevelopmental disorders, including autism spectrum disorder (ASD). Such behavioral ratings are subjective, require significant clinician expertise and training, typically do not capture data from the children in their natural environments, and are not scalable for large population screening, low-income communities, or longitudinal monitoring. The development of scalable digital approaches to standardized objective behavioral assessment is thus a significant unmet need in ASD, here addressed via machine learning and computer vision with the goal of providing scalable methods for assessing existing biomarkers, from eye tracking to movement and posture patterns, and tools for novel discovery. Our long-term goal is to develop validated scalable tools for the automatic behavioral analysis of neurodevelopmental disorders. The proposed computational project leverages results and big data derived from our previous studies (N=1,864 participants) and our recently funded NIH Autism Center of Excellence (ACE) award (N=7,436 participants). The ACE project will allow us to develop and validate our tools on several thousand toddlers recruited in Duke pediatric primary care and followed longitudinally for whom gold-standard diagnoses of ASD, attention deficit hyperactivity disorder (ADHD), developmental and language delay and extensive electronic health record (EHR) data will be available; and in a case control study of 224 age-matched groups of young children with ASD, ADHD, and typical development from whom gold-standard diagnostic, extensive phenotypic, Tobii eye- tracking, and EEG will be collected. This project aims to develop novel computational methods using these datasets, from sensing in scalable fashion behaviors such as attention and gaze (Aim 1) and motor/posture (Aim 2), to their multimodal integration (Aim 3). A unique aspect of our computational approach is the closed- loop integration of stimuli design for actively eliciting behavioral symptoms, use of consumer-grade sensors, and automatic behavioral analysis. This contrasts with the current approach of independently selecting stimuli and using expensive lab-based professional grade sensors with off-the-shelf algorithms to capture behavioral biomarkers expected from the stimuli. Our approach involves active elicitation of behavior which is also different from commonly used digital approaches that involve gathering large datasets from passive sensing, such as actigraphy monitoring of spontaneous behavior at home. Our framework results in active closed-loop sensing, where participants are engaged in short and developmentally appropriate activities on ubiquitous devices, while the sensors in the same device capture information for the automatic and quantitative analysis of behavioral biomarkers. This scalable, objective, and standardized way of stimulating, sensing, and analyzing allows the collection of large behavioral datasets for machine learning.

Recent advances in deep learning have led to many disruptive technologies: from automatic speech recognition systems, to automated supermarkets, to self-driving cars. However, the complex and large-scale nature of deep networks makes them hard to analyze and, therefore, they are mostly used as black-boxes without formal guarantees on their performance. For example, deep networks provide a self-reported confidence score, but they are frequently inaccurate and uncalibrated, or likely to make large mistakes on rare cases. Moreover, the design of deep networks remains an art and is largely driven by empirical performance on a dataset. As deep learning systems are increasingly employed in our daily lives, it becomes critical to understand if their predictions satisfy certain desired properties. The goal of this NSF-Simons Research Collaboration on the Mathematical and Scientific Foundations of Deep Learning is to develop a mathematical, statistical and computational framework that helps explain the success of current network architectures, understand its pitfalls, and guide the design of novel architectures with guaranteed confidence, robustness, interpretability, optimality, and transferability. This project will train a diverse STEM workforce with data science skills that are essential for the global competitiveness of the US economy by creating new undergraduate and graduate programs in the foundations of data science and organizing a series of collaborative research events, including semester research programs and summer schools on the foundations of deep learning. This project will also impact women and underrepresented minorities by involving undergraduates in the foundations of data science.

Deep networks have led to dramatic improvements in the performance of pattern recognition systems. However, the mathematical reasons for this success remain elusive. For instance, it is not clear why deep networks generalize or transfer to new tasks, or why simple optimization strategies can reach a local or global minimum of the associated non-convex optimization problem. Moreover, there is no principled way of designing the architecture of the network so that it satisfies certain desired properties, such as expressivity, transferability, optimality and robustness. This project brings together a multidisciplinary team of mathematicians, statisticians, theoretical computer scientists, and electrical engineers to develop the mathematical and scientific foundations of deep learning. The project is divided in four main thrusts. The analysis thrust will use principles from approximation theory, information theory, statistical inference, and robust control to analyze properties of deep networks such as expressivity, interpretability, confidence, fairness and robustness. The learning thrust will use principles from dynamical systems, non-convex and stochastic optimization, statistical learning theory, adaptive control, and high-dimensional statistics to design and analyze learning algorithms with guaranteed convergence, optimality and generalization properties. The design thrust will use principles from algebra, geometry, topology, graph theory and optimization to design and learn network architectures that capture algebraic, geometric and graph structures in both the data and the task. The transferability thrust will use principles from multiscale analysis and modeling, reinforcement learning, and Markov decision processes to design and study data representations that are suitable for learning from and transferring to multiple tasks.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Intellectual Merit: This project will for the first time provide the fundamental tools to integrate unique multimodal data toward screening, diagnosis, and intervention in eating disorders, with an initial focus on children with ARFID and related developmental and health disorders. This work is critical for enriching the understanding of healthy development and for broadening the foundations of behavioral data science. ARFID ·motivates the development of new computer vision and data analysis tools critical for the analysis of multidimensional behavioral data. The main aims are: 1. Develop and user individualized and integrated continuous facial affect coding from videos to discern affective motivations for food avoidance, critical due to the unique sensory aspects of eating disorders, and resulting from active stimulation via friendly and carefully designed images/videos and real food presentation; 2. Use data analysis and machine learning to derive sensory profiles based on patterns of food consumption and preference from existing unique datasets of selective eaters; and 3. Translate the tools developed in Aims 1 and 2 into the clinic and home to assess the capacity of these tools to define a threshold of clinically significant food avoidance, to detect change in acceptability of food with repeated presentations, and to examine and modify the accuracy of our food suggestion algorithms. Broader Impacts: The impact of this application comprises two broad domains. First is the derivation of processes, tools, and strategies to analyze very disparate data across multiple levels of analysis and to codify those strategies to inform similar future work, in particular incorporating automatic behavioral coding. Second is the exploitation of these tools to address questions about the emergence of healthy/unhealthy food selectivity across the lifespan, including recommendation delivery via apps and at-home recordings. The health impact of even partial success in this project is very broad and significant. Undergraduate students will be involved in this project via the 6-weeks summer research program at the Information Initiative at Duke, a center dedicated to the fundamentals of data science and its applications; via the co-Pl's research lab devoted to eating disorders; and via the Pl's project dedicated to training undergraduate students to address eating disorders of their friends via an anonymous app. Outreach and dissemination will follow the broad use of the developed app, both in the clinic and the general population, including the Pl's connections with low-income and under-represented bi-lingual preK. RELEVANCE (See instructions): Eating disorders are potentially life-threatening mental illnesses affecting the general population; -90% of individuals never receive treatment, in part due to lack of awareness and access. Individuals with eating disorders experience a diminished quality of life, high mental and physical illness comorbidities, and an existence marked by profound loneliness and isolation. Combining expertise in eating disorders with computer vision and machine learning, we bring for the first time data science to this health challenge. PROJECT/PERFORMANCE S1TE(S) (If addItIonal space Is needed use Project/Performance Stte Format Page)

Avoidant Restrictive Food Intake Disorder (ARFID) is a newly articulated eating disorder in the DSM-5 in which individuals are not able to consume an adequate quantity or variety of food to sustain healthy growth and de- velopment. ARFID typically onsets in early childhood, yet identification of the disorder is poor. The end result is that children often have sustained inadequate nutrition with resulting severe physical consequences and threats to optimal social and emotional development. Early intervention is essential. However, there are no em- pirically validated treatments for young children with ARFID. Children with ARFID are known to be sensitive individuals: with a low threshold for noticing internal body sensations (e.g., gastrointestinal distress) and exter- nal sensory sensations (e.g., smells) and experiencing these sensations as uncomfortable/aversive. This sen- sitivity, and the associated negative emotional reactions (e.g., of disgust, fear, sadness), may encourage avoidance of activities that cause these sensory experiences, such as eating. An intervention that could change reactions to sensory and somatic sensations to one of playful curiosity may increase approach behav- ior and food consumption. We designed an acceptance-based interoceptive exposure treatment for children to achieve this goal. We teach children and caregivers to be ?FBI Agents,? individuals who view somatic and sen- sory sensations as clues to a mystery via the use of engaging illustrative cartoon characters (e.g., Gassy Gus), body investigations that provoke intense sensations and worksheets that map sensations to meanings and ac- tions. The goal is to make somatic and sensory experiences playful ? and to promote adaptive self-awareness and food approach. The overall objective of this study is to determine whether treatment results in reduced negative emotions to somatic and sensory sensations, including those associated with food and eating, and whether this, in turn, increases food approach. This will be accomplished by a randomized controlled trial (N = 140, 70 per cell) comparing The Feeling and Body Investigators (FBI) - ARFID Division treatment, a 20-session outpatient treatment, to a control group in children (5 to 9 years of age) with ARFID. The control treatment (FAD: Family-Assisted Diet) will provide family-supported exposure. Our proposed mediator of treatment re- sponse is negative affect: a child?s facial affect in response to food presentation as measured via smartphone videos. Outcomes include changes in anthropometric measurements, nutrition variety and adequacy, and psy- chosocial functioning. Medical and psychological measurements will be assessed pre- and post-treatment, at every session, and 3-month follow-up. Deliverables include an empirically validated treatment(s); medical guidelines for the early detection of insufficient nutrition; practical tools for assessing food acceptance in the home via smartphones; and guidance about the number of food presentations necessary to facilitate acceptance. Ultimately, we will evaluate this treatment and disseminate materials via primary care to provide tools to intervene early on food avoidance.

Machine-learning algorithms may present discriminatory behavior across certain subgroups, meaning that segments of the overall population are measurably under-served by the model, rendering the decisions unfair. The most common approaches to address this challenge consider that the algorithm has access to a set of predefined protected subgroups during training, and the goal is to learn a model that satisfies a certain notion of fairness/robustness across these subgroups. Perfect fairness can, in general, only be achieved by degrading the performance of the benefited subgroups without necessarily improving the disadvantaged and protected ones. This conflicts with ethical and legal notions of no-harm fairness, which are appropriate where quality of service is paramount, for example in health. To address this, this work considers notions of fairness and subgroup robustness that guarantee no unnecessary harm is done to any subgroup. The project goes beyond this since it considers the case where the subgroups or demographics are not known a priori and might even change with time and algorithm deployment. The project brings these concepts of blind and no-harm subgroup robustness and fairness to the area of backwards compatibility, where the goal is to guarantee that new machine-learning algorithms are compatible with previous ones; and to the area of federated learning, where multiple sites share data for the sake of mutual benefit. Lastly, potential connections of the proposed blind and no unnecessary-harm subgroup robustness with causal inference are investigated.

The project first formally studies blind and no-unnecessary-harm (Pareto optimal) subgroup robustness, where the machine-learning algorithm needs to be robust to all possible subgroups of the data (given a minimal subgroup size), without necessarily knowing in advance the subgroups' defining characteristics. This is formally studied, including the tradeoffs and costs of protecting unknown subgroups and the corresponding optimization algorithm; concepts of data and optimization uncertainty are also included to model potential sacrifices a subgroup can make in benefit of others. Such formal study of blind subgroup robustness is an emerging field in the machine-learning community, and this project provides a fundamental and unifying view of it, combining theory with practice and critical information for policy makers. The project then extends the work to the area of backwards compatibility, with the goal to make all potential subgroups equally backwards compatible; and to federated learning, where the subgroup fairness and robustness is considered both across the silos/participants and inside each silo itself. Finally, thanks to the close mathematical connection between invariant features and causality, the project further considers this proposed unifying framework of blind subgroup robustness to study connections between the automatically discovered critical subgroups, their features, and causality. Health applications provide a unique testbed for the frameworks developed here.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.