Aude Oliva - US grants

DESCRIPTION (provided by applicant): The application addresses a fundamental tension in visual cognition: how are human observers able to recognize the semantics of a complex real world scene image at a glance? Since the seminal work of Mary Potter, a number of experimental studies have demonstrated that we identify a surprising amount of information from a single glance at a scene. We can recognize its semantic category (e.g. a street), some objects and regions (e.g. a red car on the left) and other characteristics of the space that the scene subtends in the real world (e.g. perspective). This information refers to the "gist" of the scene and can be identified as quickly and as accurately as a single object. The principal aim of this project is to define the perceptual content of the image information acquired during a glance at scene photographs. In this application, we consider the case of the image being conceptualized in short-term memory. We aim to propose an experimental paradigm that allows the comparison of the quantity of information common to pairs of scene images. The research program introduces an innovative image similarity measure that defines the exact quantity of spatial and spectral components common to images that share the same semantic category. The results of the proposed research program shall provide researchers in visual cognition with the knowledge about the quantity of image information that, on average, adult human observers are seeing and remembering within a brief exposure to a novel picture. The research should demonstrate that the quantity of information varies with the task the observer has to perform. More precisely, the study aims to explore the information that human observers may use when recognizing a scene at different levels of abstraction (its superordinate level, its basic-level and its subordinate level of description).

Computer vision started with the goal of building machines that can see like humans. Nowadays, computer vision has expended to numerous applications such as image database search in the world wide web, computational photography, reconstruction of three-dimensional scenes, surveillance, assistive systems, vision for graphics and nanotechnology, etc. More domains and applications keep arising as computer vision technology develops.

The goal of the workshop in Frontiers in Computer Vision is to bring together national and international experts, from academia and industry, to identify the future impact of computer vision on the economic, social, educational and security needs of the nation and outline the scientific and technological challenges to address the issues: how can computer vision build on the success and enthusiasm of its growing participants? how can the academic community make connections to industry? how to better foster scholarship and improve communication both within computer vision itself and to related disciplines and application areas? How can computer vision best interact with related fields? how can the importance and promise of computer vision be communicated to the general public?

The deliverables of the event include, among others, videos of the presentations available on the Frontiers in Computer Vision website, together with a roadmap to outline the scientific and technological challenges to address.

US-French Collaboration in Computational Neuroscience, Paris, November 29-30, 2011

This award supports a US-French workshop, led by Aude Oliva and Alain Destexhe, on binational collaboration in computational neuroscience. The workshop builds on interests by NSF, Agence Nationale de la Recherche (ANR), and other agencies in collaborative research in this rapidly developing area of research. The workshop will explore the intellectual opportunities, educational and economically relevant impacts, and practical considerations needed for US-French collaboration to be successful. The workshop will be attended by US and French researchers, and representatives from US and French funding organizations. A report from the workshop will be made available at http://www.nsf.gov/crcns.

As Lewis Carroll famously wrote in Alice in Wonderland - It's a poor sort of memory that only works backwards-. On this side of the mirror, we cannot remember visual events before they happen; however, our work will help predict what people remember, as they see an image or an event. Our team of investigators in cognitive science, human neuroscience and computer vision bring the synergetic expertise to determine how visual memories are encoded in the human brain at milliseconds and millimeters-resolution. Cognitive-level algorithms of memory would be a game changer for society, ranging from accurate diagnostic tools to human-computer interfaces that will foresee the needs of humans and compensate when cognition fails.


The project capitalizes on the spatiotemporal dynamics of encoding memories while providing a computational framework for determining the representations formed from perception to memory along the scale of the whole human brain. A fundamental function of cognition is the encoding of information, a dynamic and complex process underlying much of our successful interaction with the external environment. Here, we propose to combine three technologies to predict what makes an image memorable or forgettable: neuro-imaging technologies recording where encoding happens in the human brain (spatial scale), when it happens (temporal scale), and what types of computation are performed at the different stages of storage (computational scale). Characterizing the spatiotemporal dynamics of visual memorability, and determining the type of computation and representation a successful memorability system performs is a crucial endeavor for both basic and applied sciences.