John C. Duchi - US grants

People have different ways of learning and thinking, termed cognitive styles. Past research in website design has shown that there is a link between cognitive style and user behavior. This project takes this promising foundation and applies it to the design of physical products. This EArly-concept Grant for Exploratory Research (EAGER) project investigates whether or not it is possible to use sensor data and morphing algorithms, a type of learning algorithm, to design a faucet that can "know" what a person wants to do, and how they prefer to do it, via an underlying relationship between cognitive style and behavior. If so, can the faucet be designed in a way that its behavior is adaptable and pleasing to distinct cognitive styles, while also reducing water consumption. Faucets and showers account for 20% of household water usage, yet have received no "smart" design improvements to curtail water use. On the contrary, research shows that current automatic on/off faucets use more water than conventional faucets. If successful, this research will advance the design of household appliances that decrease water consumption.

The project objective is to create a design method that uses morphing algorithms to design generative, customized product behavior that responds to the user's cognitive style and the task they are performing. This involves: (1) Reworking existing morphing/learning algorithms to make them generate a customized product behavior, instead of serving-up predetermined design permutations; (2) Creating a protocol to identify meaningful independent variables (sensor data) that serve as the parameters for controling morphing; (3) Incorporating feedback from users, in the form of faucet manual adjustments, to the behavior updating process; and (4) Balancing exploration of the behavior space and exploitation of knowledge gained. The sensor data used in this initial research will be simulated based on a pilot study. The research advances the state of the art in learning algorithms, increasing their usefulness in design by allowing for continuous-space design exploration in response to manual human-in-the-loop user interaction behavior. If successful, it will result in a physical product that is capable of testing the relationship between cognitive style and user interaction. This product will be used in future human-subject experiments, potentially building new cognitive models of user/product interaction.

Modern techniques for data gathering?arising from medicine and bioinformatics, internet applications such as web-search, physics and astronomy, mobile data gathering platforms?have yielded an explosion in the mass and diversity of data. Concurrently, statistics, decision theory, and machine learning have successfully laid a groundwork for answering questions about our world based on analysis of this data. As more information is collected, classical approaches for inference and learning are insufficient, as additional concerns arise?computational resources, privacy considerations, storage limitations, network communication constraints? outside of statistical accuracy. This prompts a basic question: how can multiple criteria be balanced while maintaining statistical performance?

To bring statistics and machine learning into closer contact with other desiderata, this research involves the development of procedures that trade between scarce resources in principled and optimal ways. Such trade-offs have been difficult to characterize, as current tools for providing fundamental limits (such as information theory in communication) do not connect disparate areas. Three concrete sub-areas serve as bases for this research. The investigators study the interplay of computing with learning, estimation, and optimization by connecting notions of computation?such as memory accesses or synchronization in distributed systems?to data analysis tasks. Second, the research investigates adaptive and robust procedures?and associated statistical costs?that will become more important given increasingly long-tailed and messy data. Thirdly, the investigators study privacy in estimation, using information and decision-theoretic tools to characterize the tensions between statistical accuracy and sensitive data disclosures. Combined, these lay the groundwork for a theory on the use of data in the face of constraints, along with a functional and practical understanding of procedures that balance scarce resources against statistical accuracy.

The application of machine learning, in fields from medicine to mobile data gathering platforms, has substantial promise. Yet as data comes from a greater variety of sources in an ever-shifting world, how can one trust that machine-learned systems have not simply fit some strange idiosyncrasies they observe? This project develops methods for machine learning so that such systems are not brittle, sensitive to tiny changes in collected data, or likely to make critical mistakes on rare populations. With the growing importance of data analysis in science, industry, and healthcare, principled and practical approaches to robustness, safety, and calibration have immediate and wide-ranging effects. A major goal of the project is to provide decision makers with trustworthy predictions from machine-learned models. A second goal is pedagogical: with the meteoric rise of machine learning, there is a missed opportunity to educate students, researchers, and engineers to give them the ability to actually build trustworthy systems; this project aims toward a curriculum around such challenges.

This project develops robust learning procedures in effort to build trustable machine learning. Three concrete thrusts underpin the work. The first builds off of the investigator's work in distributional robustness, which fits models to maximize performance on populations near enough to available data. The second is to use data creatively and correctly; this entails using the data to define robustness, understand method sensitivities, use unlabeled (cheap) data to build more robust representations, and construct data-based regularization. The third targets confidence and calibration, building models that provide assumption-free valid predictions. In this case, the aim is to seek predictors with calibrated confidence, building out of conformal prediction, which modern learning methods emphatically do not provide. More generally, distributional shifts challenge statistical machine learning methods, and the project aims for new validation and testing methodologies to understand such shifts, identify situations where methods are sensitive to changes in underlying data, and to allow valid confidence in predictions even in changing environments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.