Alyosha Molnar - US grants

DESCRIPTION (provided by applicant): The overarching goal of this project is to develop a new approach to capturing information about the three- dimensional structure of fluorescent samples. The hardware we develop will be able to capture this information without specialized optics, or indeed without any optical apparatus at all beyond an LED and our chip. As such, this technology has the capacity to reduce the size and cost of imaging-based assays by orders of magnitude. Deployed as part of highly parallel and/or portable instruments, this sensor will dramatically expand the types and amounts of useful data that can be gathered. The specific aims of this project, discussed below, are: 1) development of novel opto-electronic hardware (a CMOS chip) using existing, scalable semiconductor manufacturing;2) development of data-processing tools for the reconstruction of fluorescent structure from chip output;3) demonstration of this system for both static and dynamic imaging of fluorescent cells or tissue;and 4) provision of a unique research experience for scientist- engineers in training. The key element that will enable chip-scale lensless imaging is an angle-sensitive pixel manufactured entirely in CMOS. Such a pixel has been demonstrated by the PI's lab, detecting not only light intensity, but also its incident angle. These pixels make use of near-field diffraction patterns generated and filtered by local gratings to only pass light at certain incident angles. This light is detected by local photodiodes, just as in a normal CMOS imager. By combining multiple sub-pixels with different preferred angles, one can construct a full angle-sensitive pixel that is of a similar scale to pixels in existing imagers. The gratings are constructed using the metal wiring layers present in any modern semiconductor process, and the photodiodes use standard semiconductor junctions. Thus arrays of angle-sensitive pixels can be constructed entirely using existing CMOS manufacturing processes identical to those used to build other integrated circuits and imagers. Such chips, can be manufactured at extremely low cost (<$5 apiece) providing a massive reduction in the cost and size of image-based assays. Simulations of arrays of angle-sensitive pixels indicate that they will be able to localize multiple fluorescent sources such as GFP tagged cells distributed in 3-d space. We will design such arrays (to be manufactured through MOSIS) and deploy them in simple micro-fluidic packages to demonstrate their utility in imaging cell and tissue cultures and in high-throughput flow cytometry. Our final goal is to build a centimeter-scale instrument able to image the three-dimensional structure of a fluorescent sample at high frame rates with a manufacturing cost of less than $10. PUBLIC HEALTH RELEVANCE: The goal of this project is to develop very small, very cheap instrument able to image the three- dimensional structure of a biological sample, replacing microscopes for many applications. This capability would enable filed-deployable imaging systems for doctors and scientists away from the lab. Also because of its low cost and small size, this instrument will enable massive parallelization of imaging-based assays, enabling fast screening of large numbers of tissue samples or cell cultures in, for example, drug discovery.

The objective of this research is to develop radios for multi-function bird tracking tags. Each radio tag will support geo-localization, in-flight telemetry and networking. The approach is to develop a passive-mixer-first radio architecture that connects highly flexible baseband circuitry to the antenna through a "transparent front-end."

State-of-the-art radios provide programmable gain, bandwidth, and center frequency, but have fixed antenna interfaces, dramatically reducing their flexibility. The proposed ?transparent front-end? allows direct interaction between highly controllable baseband circuitry and variable, poorly controlled antennas. Thus, all important properties of the radio are made programmable, including previously fixed properties such as impedance matching, allowing on-the-fly reconfiguration. Furthermore, this flexibility comes with little or no cost in terms of power consumption or performance. By developing the theoretical and practical aspects of the design of transparent front-ends, this project will complete a multi-decade trend from fixed, single function radios to fully flexible multi-function transceivers.

Because transparent front-end radios enhance performance while reducing cost and size, they are likely to find use in a variety of scientific and commercial applications. The project also provides a showcase of different aspects of wireless technology which will be used in educational demonstrations. By permitting link-ups between RF tags and commercial handheld devices, this project will also add a new dimension to citizen science in field ornithology. The research program will support graduate research and undergraduate design projects and outreach activities to K-12 students.

Radio-enabled tracking tags have dramatically enhanced our knowledge of wildlife behavior over the last 3 decades, and new types of tags are being developed which capture and transmit dramatically more information about their behavior. Advances in microelectronics have also reduced the size and increased the lifetime of tags, increasing the number of species that can be tagged. Unfortunately these various tags use a wide variety of frequencies, coding schemes, and communication protocols, requiring an entirely new base station for each new tag. This makes new deployments more expensive, disallows sharing of monitoring stations between overlapping studies, and requires that that biologists in the field carry a base station for every tag type in use.

The goal of this project is to develop a general-purpose radio transceiver for behavioral ecologists and others that can be easily configured to communicate with any existing wildlife tracking tag, as well as with foreseeable future designs. Our approach is based on software defined radios, where most of the parameters of the radio are configured in software, allowing a single piece of hardware to cover multiple types of tags. In this project we will develop working prototypes able to cover at least 3 different existing avian tags, and show them working in the field. We will also develop a mode where signals can be captured, but not decoded, to allow tag signals from different researchers to be captured by the same physical basestation, and then distributed to the various researchers to be decoded. Finally, we will ensure that the resulting outputs are in a format that allows smooth upload of captured signals to databases such as MOVEBANK.

To ensure that our 'universal base stations' are available to other researchers, we will post on the internet all board designs, parts lists and software we develop, as well as documentation of all designs and software. Our goal is that UBS designs be accessible not only to researchers, but also to amateur birders and other 'citizen scientists', allowing them to contribute to research, as well as enhancing their birding experience (or example, by allowing them, after detecting a tagged bird, to look up 'their bird' in a database and learn more about it's particular behavior and movement. To further this goal, we will attempt to develop a smart-phone interface to our UBS, to allow birders in the field to capture and upload data from tags they encounter via their phones.

While this work will provide a valuable tool for biologists, both professional and amateur, it will also provide valuable educational experience for engineering students. Parts of this project will be incorporated into class projects, and much of the design work required will be completed by undergraduate and Masters of Engineering students completing as design projects required for graduation. Because this project requires teamwork, has specific concrete goals and specifications, and clear end-customers (field biologist who are collaborating on this project), this project will provide invaluable experience in preparing students planning on working in industry.

For more details, see: http://molnargroup.ece.cornell.edu/research.html

The objective of this project is to develop integrated, low cost systems able to capture and characterize light from 3-D scenes while minimizing or eliminating the need for off-chip computation.

The intellectual merit of this research is in understanding the capabilities and limitations of imaging systems using diffractive angle-sensitive pixels to analyze the light field. This includes: 1) Developing models of how scene statistics are reflected in the output statistics of angle sensitive pixel arrays, and so developing algorithms to extract useful information about the scene, such as depth, 3-D directional motion, and object recognition. 2) Understanding how to optimize arrays of angle sensitive pixels to encode scenes at various focal depths, as well as how to best deploy signal selection, conditioning and processing circuits to take advantage of these arrays? signal properties when encoding real scenes. 3) Investigating new integrated diffractive structures that enhance ASP quantum efficiency and spatial resolution, and that filter the light field in new ways.

The broader impacts of this research will include enhanced imaging systems with applications in security, automation, health care, and scientific research, from wildlife tracking to microscopy. This work will also impact the commercial sphere by providing new capabilities in consumer electronics, helping to boost economic growth. These impacts are increased because these systems use standard CMOS manufacturing, and so can be mass-produced at very low size and cost. This research provides an easily-understood example of low-level physics driving useful high-level function, providing compelling examples for recruiting and inspiring future engineers.

Diffractive masks and algorithms for light field capture
PIs: Ramesh Raskar, MIT Media Lab Alyosha Molnar, Cornell ECE

Advanced imaging and display technology requires integrated, low cost systems able to efficiently capture and characterize light from 3-D scenes. In particular, a 3-D scene can be described by the collection of light rays it generates, called the light field. This research combines concepts from mask-based light-field imaging with angle sensitive pixels (ASPs). While mask-based light-field capture is much better understood mathematically, and masks are cheaper to manufacture and more easily modified on-the-fly, diffractive ASPs provide smaller, denser light field sensors, and provide naturally compressible outputs. This project combines the physics and signal processing of these approaches to enable optical imaging systems that capture more information than normal cameras while reducing the system's complexity. This work broadly impacts diverse applications spanning consumer imaging and displays, machine vision and automation, scientific/medical imaging and displays, robotic surgery, surveillance and remote sensing.

3-D images and video can be captured by measuring the combined spatial and angular distribution of light (the light field). This research combines two techniques for light-field capture: mask-based light-field imaging and diffractive angle sensitive pixels (ASPs). A critical element of this work is the development of a mathematical framework that maps between conventional geometric light fields and the diffractive optics upon which ASPs rely. A second element is constructing hybrid systems based on this mathematics, leveraging diffractive effects in mask design, and combining masks with ASPs in single light-field cameras. This work also combines formalisms in existing light field methods with knowledge about real 3-D scene statistics to develop optimal (in the sense of usability and compressibility) basis sets for sampling and encoding the light-field. All of these aspects combine to reduce the size, cost and complexity of light field cameras, while simultaneously enhancing their capabilities.

Objective: The objective of this multidisciplinary program is to develop disruptive Radio Frequency (RF) technologies that provide significant spectral efficiency gains at the physical layer, by leveraging recent advances in physical layer interference management and integrated receiver design.

Intellectual merit:
The intellectual merits of this project is composed of the following three thrusts: (i) RF Front-End Signal Projection and Cancellation for Enhanced Dynamic Range, (ii) Controlled Reflectors for Receiver-Centric Interference Management, and (iii) Fundamentals of Receiver-Centric Interference Management. The first thrust utilizes information about the structure of the interference to suppress it early in the receiver chain, shielding subsequent stages from full dynamic range requirements of the system. The second thrust develops RF receivers which double as controlled, coordinated reflectors, and are able to alter the effective channel gains to nearby receivers. The third thrust develops new interference management strategies in the light of the aforementioned new capabilities of transceivers and studies the fundamental limits of spectral efficiency under such technologies.

Broader impacts:
By developing collaboration and communication between two fields of "information theory" and "RF Integrated Circuits", this program opens new horizons in both fields. This project impacts the nation's economy not only by enabling greater network capacity, but also by spurring technological advances that can easily be transferred to the wireless and electronics industries, driving economic growth in those areas. This project also provides opportunities for research experience for undergraduates and includes several outreach activities.

The explosion in wireless data usage across the world has been one of the great economic drivers of the last decade, and the global need for cheap, high-data-rate wireless access is expected to continue to grow rapidly for at least another decade. Non-reciprocal components, such as circulators and isolators, enable new wireless communication paradigms such as full-duplex wireless that are otherwise not feasible and promise to significantly enhance wireless data capacity. However, non-reciprocal components today are almost exclusively realized through the magneto-optic Faraday effect, requiring the use of ferrite materials that are expensive, bulky and incompatible with the silicon-based integrated circuit technologies that power the wireless and computing revolutions. This proposal will devise, analyze and experimentally demonstrate new multi-physics approaches based on spatio-temporal modulation that enable the breaking of reciprocity at radio frequencies (RF), and the realization of compact and low-cost RF non-reciprocal components integrated in commercial silicon-based technologies. Broadly, this research will have a direct and critical impact on the societal need for enhanced access to wireless data by expanding the range of accessible RF spectrum, enabling new schemes for more efficiently sharing existing spectrum, and reducing the size and cost of nonreciprocal components, making wireless devices accessible to a larger portion of the population. In terms of outreach, this project will also provide opportunities for the education and engagement of students across the educational continuum from kindergarten through graduate school (K-12, undergraduate and graduate), leveraging existing programs at Columbia and Cornell, in collaboration with the Liberty Science Center in NYC and a number of coordinated outreach and diversity programs.

Recent research has revealed that introducing time variance into a material or system enables the breaking of reciprocity. While there are no fundamental performance limits associated with time-varying non-reciprocal systems and components, existing spatio-temporal modulation approaches for the realization of non-reciprocal components such as circulators have been fraught with challenges in insertion loss, size or linearity. While traditional spatio-temporal modulation approaches have relied on the variation of permittivity in dielectric media, the key insight in this proposal is the fact that semiconductor systems offer another material property that can be more powerfully modulated ? namely conductivity. Based on this insight, this project will pursue various multi-physics approaches to achieve non-magnetic RF non- reciprocity in silicon-based integrated circuit technologies. Blending electronics with acoustics and optics in an integrated setting, this project will demonstrate high-performance non-reciprocal circulators and isolators at RF, millimeter-wave and terahertz frequencies. This project will also investigate synthetic RF media supporting topologically protected non-reciprocal modes of propagation from both a theoretical (i.e. mathematical) and experimental perspective, with applications in non-reciprocal antenna interfaces for emerging wireless communication paradigms such as large-scaled phased arrays and massive Multiple-Input-Multiple-Output (MIMO) systems.

Summary Our goal in this project is to develop a new class of electrical recording device that complements and piggy- backs on cutting edge imaging technologies. Whereas multi-electrical recording has provided detailed measurements of neural activity with high temporal precision, it is also invasive, provides relatively low spatial resolution, and provides little information about the identity of measured neurons. Optical imaging techniques, conversely, provide very fine spatial resolution, easing neural identification, but at the cost of significantly worse temporal resolution, and with the requirement of either chemical (through fluorescent dyes) or genetic modification of the tissue. In order to better bridge these two modalities, we envision developing untethered Microscale Optoelectronically Transduced Electrodes (MOTEs) which combine optoelectronic elements for power and communication with custom CMOS circuits for low-noise amplification and encoding of electrical signals. Each MOTE will be powered by optically stimulated micro-photovoltaic cells and will use the resulting 1-2µW of electrical power to measure, amplify, and encode electrophysiological signals, up-linking this information optically by driving an LED. MOTEs will avoid many of the problems associated with standard wire- and shank-based electrodes, where most of the volume of the implanted electrode, and so most of the tissue damage it does, stems from the long rigid shank that connects electrode sites to external electronics. To be most useful, MOTEs' photovoltaics will be designed to harvest power from optical stimuli of the same wavelengths and intensities as are used in stimulating fluorescence when imaging neural activity. Similarly, the LED used for uplink will be designed to emit light at wavelengths and intensities consistent with those detectable by a fluorescent imaging system. These choices will allow the both down- and up-link of optical signals to be handled by existing imaging systems with minimal modification. By employing a pulsed stimulation (as is used in multi-photon systems) and appropriately encoding and timing up-linked LED pulses, fluorescent and MOTE emissions can be segregated into adjacent sub-microsecond time bins. This combination of optical compatibility and temporal multiplexing will allow simultaneous imaging and electrical recording of neural activity from the same volume of neural tissue, using the same optical imaging and recording systems. This simultaneous, heterogeneous measurement capability will enable a much wider range of experiments and studies of neural activity than are presently possible.

The goal of this project is to leverage and improve upon the capabilities of honey bees as agricultural pollinators by incorporating them into Bio-Cyber Physical systems. Rapid advances are needed to aid a dwindling agricultural workforce, increase crop yield to sustain the growing population, and provide targeted crop care to limit the need for broad pesticide treatments. These challenges may well be addressed by autonomous mobile robots and sensor networks; unfortunately, agricultural landscapes represent vast, complicated, and dynamic environments that complicate long term operation. In contrast, social insects are capable of robust sustained operation in unpredictable environments far beyond what is possible with state-of-the-art artificial systems. Colonies of honey bees are of particular interest in this project, because they are the premiere agricultural pollinator bringing in over $150 billion annually. The U.S. has an estimated 2.4 million colonies, many of whom travel the country every summer to help pollinate monocrops such as almond and corn. A colony causes pollination by dispatching tens of thousands of scouts and foragers to survey and sample kilometer-wide areas around their hive. Thus, the colony as a whole accumulates vast information about the local agricultural landscape, bloom and dearth -- information that would be very informative if available to farmers and beekeepers.

This project will leverage social insects as environmental indicators by piggybacking on their naturally existing capabilities. It involves sensors to record where bees focus their foraging activity, and mechanisms to stimulate additional foragers. This project will impact: 1) ultra-low power electronics and sensing, 2) probabilistic inference from large scale distributed data sources, 3) feedback control of biohybrid systems, and 4) gains to apiculture and entomology. The proposed bio-hybrid technology may further inform models of how bees forage in natural versus cultivated areas, and may lead to new insights on design of agricultural multi-use landscapes for improved yield. Overall, this research will improve engineers' understanding of Bio-Cyber Physical Systems able to monitor and affect the environment, and may be applicable to scenarios including search and rescue, detection of chemical spills, and targeted pollination.

To harness the capabilities of a bee colony while still providing control and sensing, the proposed work specifically involves 1) novel submillimeter flight recorders with visual scene capture and analysis, thermal and mechanical sensors, a clock, storage, processing, photovoltaic chargers and short range communications; 2) algorithms and models to estimate foraging maps, relying on bee motion models and feature extraction, merging probability density functions of observed landmarks from thousands of flights; and 3) feedback control via a bee-mimicking shaker device to recruit foragers, in turn eliciting data collection and pollination, e.g. during brief spouts of bloom that would otherwise go unnoticed by the colony. This research represents a transformative step towards a new frontier in Bio-Cyber Physical Systems, improving upon the abilities of social insects to sense and interact with the physical world, while providing data acquisition and control on par with explicitly engineered systems.

As the demand for wireless services increases and the usable spectrum becomes ever more crowded, wireless systems need to become more robust against interference from many other signals. Radio receivers form the last line of defense, protecting wireless systems from today?s increasingly dynamic and densely occupied spectral environments. This project will develop a novel radio receiver architecture capable of operating across a large portion of the wireless spectrum while simultaneously being capable of adaptively suppressing interferences as they arise. Since interference may change as a function of time and location, an algorithm will be developed to help the receiver adaptively adjust its response to one or more of these interferers so that the system can take advantage of the wireless spectrum whenever and wherever there is a need. Recent explosive growth in internet usage have brought to light humanity?s increasing dependence on wireless access and the significant role wireless radio receivers have in enabling the continued expansion of connectivity. This project has specific plans to educate and train rising engineers, at both the graduate and undergraduate level, to think holistically about the components and operation of wireless systems and establish robust receivers for the future. Specifically, PIs will pilot a new seminar course needed to succeed in a doctoral degree program, which will include managing advisor-advisee relationship, reading and writing research papers, giving effective research presentations, and pursuing a career after graduation. PIs also have plan to partner with Diversity Programs in Engineering at Cornell to recruit incoming doctoral underrepresented minority (URM) students from across all engineering disciplines for the one-hour seminar each week during the Fall semester.

The project will horizontally integrate signal processing and algorithm development, circuit design and optimization, and RF component design and tuning to create a new class of receivers able to identify, adapt to, and suppress interference effects while maintaining maximum frequency agility. Research will focus on three integrated and interdependent thrust areas. Design of receiver front-ends that use passive networks (of inductors, capacitors, and other electromagnetic elements) to diversify the inputs from one or more antennas into a larger number of output taps, which then feed into a bank of reduced-power sub-receivers. Such a radio will be able to receive signals from a wide range of frequencies, while providing enough measures of both signal and interference that the byproducts of that interference can be separated from the signals using digital signal processing. This will involve both developing the required circuit theory and optimization tools and designing working prototypes at the printed circuit board and integrated circuit level. Development of digital-domain algorithms to provide control feedback to the front-end to enhance the required diversity for proper suppression. Development of adaptive RF magnetic devices to provide real-time tunability of the passive network. This will involve magnetic material and device development, and require close interaction with the circuit and algorithm designs, to best understand the optimal balance between different component trade-offs, such as between tuning range, component quality factor, and frequency of operation. The proposed new receivers have the potential to enable significant enhancement in adaptive interference mitigation and improve the robustness of future wireless systems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.