Ralph Etienne-Cummings - US grants

Phase distortions due to inhomogeneities in the optical path severely limit the perforinancc of a large class of optical systems for ground-to-ground and space communications, imaging through the atmosphere, medical laser beam focusing, among others. Demands on increased spatial resolutions and larger bandwidths call for an integrated approach to adaptive optics that modulates the wavefront in parallel at microscopic scale.

This collaborative effort combines expertise in adaptive optics, analog parallel very-large scale integrated (VLSI) niicrosys-tems, microfabrication and liquid-crystal molecular systems to create a new generation of adaptive micro-optical systems for high-resolution wavefront correction, with over 10,000 fully autonomous control elements integrated on a single, hybrid opti-cal/electronic chip. Autonomy is essential for high-bandwidth operation, and is obtained by integrating all adaptive functions directly on-chip.

At the architectural level, model-free adaptive control is implemented using parallel perturbation stochastic gradient descent optimization of an arbitrary, externally provided metric of system performance. At the physical level, high-speed wavefront control at micro-scale resolution is obtained by integrating a new type of fast nematic liquid-crystal (LC), operating at kilohertz- range bandwidths, onto the adaptive control chip. Silicon-on-sapphire (SoS) technology with ultra-thin silicon (UTSi) transis-tors provides a high-quality, low-noise, transparent active medium for high-density optical and electronic integration. We will investigate microscale structures of LC material sandwiched in between two transparent SoS wafers, implementing arrays of phase modulators with active electrodes implementing the adaptive algorithms in parallel. directly interfacing with the wave- front. The architectural and technological innovations combine to yield a projected system performance in excess of 108 control updates/sec. at least a factor 1,000 better than presently existing adaptive optics systems in speed, density and cost.

This program integrates research and education in a sequence of project-intensive courses, where teams of graduate and undergraduate students learn to design. prototype and test adaptive optics co-processors, implemented in analog VLSI and fabricated through MOSIS. The adaptive co-processors will be configured to externally control a variety of fast LC and other spatial light phase modulators, available for experimentation at the Army Research Laboratory (ARL). In addition, we will make use of full-size UTSi SoS wafers provided by Peregrine Semiconductor, custom-fabricated in a special arrangement with Hopkins, to prototype a fully integrated version of consistent optical quality. The already polished SoS wafers will be post-processed at the JHU Microfabrication Laboratory and at Boulder Nonlinear Systems. Inc.. to pattern and deposit fast nematic LC in contact with SoS for fast spatial light phase modulation. The prototyped adaptive micro-optical systems will be experimentally demonstrated on various adaptive optics and imaging tasks including laser beam focusing and stabilization for optical communications.

ABSTRACT

Although digital signal representation has become almost universal, there are still many areas where an analog representation is required to interface with an analog world or to meet various other objectives such as power dissipation, frequency, or cost. In these domains analog signal representation is essential for many input modalities such as instrumentation, sensor interfaces, and communications. Likewise, there are related output applications, such as biomedical actuation and industrial control. In addition, the needs of wireless and fiber-optical communication have reinvigorated analog design. However, there are serious problems concerning how to keep these analog components on a reasonable scaling curve as Moore's law continues unabated in the digital domain, and in integrating analog representations into large, complex digital systems ("system on a chip").

The purpose of this proposal is to study a new approach to representing analog signals that we believe will integrate more cleanly into these deep submicron, single-chip systems. Today analog signals are almost exclusively represented by current or voltage quantities. Our proposal is to borrow a page from neuroscience and to use the Inter-Pulse-Interval (IPI) between single-bit, asynchronous pulses to represent analog quantities. We are proposing to develop a mixed-mode analog/digital cell library and design methodology based on IPI representation and the associated computation elements, and to engineer a case study to illustrate the outcomes.

As we move to deep submicron and then on to the nanometer/molecular devices, the problems that digital encounters with scaling, such as threshold inconsistency, subthreshold currents, hot-electron effects, doping variability, substrate coupling, and transmission line and complex cross-talk effects, are even more serious for analog circuitry. IPI representations will provide significantly better immunity to these effects, as well as to the more traditional process, temperature, and reference voltage variations. For most applications, pulse based analog systems will require less power. There are numerous advantages to using pulses or pulses to communicate. They are significantly more immune to noise. An approximate analogy would be that of AM versus FM radio signal representation. The outcomes of the proposed research are

Create a library of basic communication, computation (arithmetic and logic) and conversion building blocks;
Design, implementation and testing of a case study using the derived building blocks and methodology;

Document an IPI-based design methodology

This Sensor proposal focuses on the development of sensory information
processing front-ends for a minimally-attended visual surveillance
network. We intend to use mixed-signal hardware computation methods in a
system-on-a-chip architecture. Image processing algorithms are
implemented at the focal-plane, using compact circuits that operate at
microwatt power levels to minimize battery weight and form factor.
Furthermore, the sensory front-end must demonstrate operational autonomy
(i.e. in decision making) and robustness (i.e. to changes in
environmental conditions).
Intellectual Merit: The proposed system will replace traditional
computer vision systems (cameras, digitizers and processors) with a
computational sensor. Using wide-dynamic range imaging, with local gain
control, circumvents the limitations of standard cameras. Spatiotemporal
feature extraction is used to highlight moving targets. The shape formed
by the features is identified using kernel learning in silicon; alarms
are generated based on the similarity of the targets to preprogrammed
shapes. Without these types of smart, ultra-low power, compact imaging
and computational microsystems, practical sensor networks for visual
surveillance may not be realizable.
Broader Impact: The sensors are primarily intended for surveillance of
large, remotely located areas, where limited manpower is available, e.g.
border patrolling. Their low power and small size obviates a variety of
mobile applications.
The multi-disciplinary nature of this work will result in the
development a pipeline of students, educators and researchers with the
broad skills required to succeed in modern high technology industry and
academics. Their education will be rounded with exposure to issues in
homeland security, privacy rights and international law.

Fellows will rotate through an internship with several distinct K-12 schools that serve disadvantaged children. Master Teachers who have been involved in research at JHU or who are currently participants in a Native American Math Science Partnership will work with the Fellows: to develop pedagogical skills to teach children with different learning styles; to enhance the content knowledge of district teachers; and to facilitate the creation of standards aligned content based on cutting-edge research. Each project will focus on topics from Environmental Engineering and Geography (EEG), including geology, hydrology, ecology, geomorphology, environmental chemistry, human factors (relations between human activities and environmental change). The Fellows will relate each project: (1) to standards, such as the American Association for the Advancement of Science.s Project 2061, Flow of Matter in Ecosystems;. (2) to compelling social issues as global climate change, and preservation of ecosystems; and (3) to the utilization of materials by students with physical limitations.

The intellectual merit lies in the potential to advance our understanding of how formal university-K-12 partnerships can improve teaching and learning by delivering challenging and relevant science, technology, engineering and mathematics (STEM) content to traditionally disadvantaged K- 12 students with a wide range of learning styles. A team of eminent scientists, future STEM faculty, leading K-12 teachers, and education experts will work as teams to use engaging EEG content and pedagogy to overcome barriers such as: students. unstable learning environments, curriculums that are in transition due to educational policy shifts, the lack of accommodations for students with physical disabilities, and the scarcity of resources.

Broader impact will be achieved through the development of sustainable K-12 curriculum and laboratory modules deployed by teachers prepared to deliver advanced, multi-disciplinary, and multi-contextual material spanning the STEM disciplines. An independent evaluator will gather information and report the findings on the feasibility of using EEG activities and instructional materials to improve the academic achievement of disadvantaged students with diverse needs.

The Telluride Workshop on Neuromorphic Cognition Engineering
Neuromorphic engineers design and fabricate artificial neural systems whose detailed architecture, design, and computational principles are based on those of biological nervous systems. Over the past 12 years, this research community has focused on the understanding of low-level sensory processing and systems infrastructure; efforts are now expanding to apply this knowledge and infrastructure to addressing higher-level problems in perception, cognition, and learning.
The annual three-week intensive Workshop (held in Telluride, Colorado) consists of background lectures (from leading researchers in biological, cognitive, computational, engineering and learning sciences), practical tutorials (from state-of-the-art practitioners), hands-on projects (involving established researchers and newcomers/students), and special interest discussion groups (proposed by the workshop participants). For researchers in this community, this is the premier workshop for training students, initiating collaborations, and in-depth discussions on scientific issues.
In this workshop and through the Institute for Neuromorphic Engineering (INE), the mission is to promote interaction between senior and junior researchers; to educate new members of the community; to introduce new enabling fields and applications to the community; to promote on-going collaborative activities emerging from the Workshop, and to promote a self-sustaining research field.
Specific Goals for the period of 2007-2012: While there is no question that the Workshop has been very successful in its mission, three new challenges have been identified for the Workshop: 1) with a rapidly expanding community in both the U.S. and Europe, the Workshop experience needs to reach more people without increasing the size of the Workshop, 2) as larger and more challenging projects are tackled, more opportunities for group interactions are needed throughout the year, and 3) as more complex questions are asked at the system-level, more voices from cognitive neuroscience are needed.
To meet these new challenges, a new version of the Workshop is envisioned with: 1) an expanded theme to focus on Perception, Cognition, and Learning, 2) an expanded constituency, educational mandate and research focus to incorporate members of the NSF Science of Learning Centers (SLC), 3) to create a two-part Workshop series (to allow yearlong collaborations and deeper investigation into large scale projects), one held in the U.S. and funded by U.S. resources and the other held in Europe and supported by European resources and 4) a modified Workshop schedule to emphasize training at the beginning of the workshop to provide a needed focus on education for both beginners and experts alike. The infusion of new researchers (from the SLCs) that focus on learning at multiple scales (from synapses to classroom) will provide the needed knowledge, new collaborations, and new perspectives to move the community towards cognitive-level neuromorphic systems.
Broader impact of the Workshop to the public: The Telluride Neuromorphic Cognition Engineering Workshop will continue its tradition of public interaction. In particular, there will be a continuation of the educational program for K-12 students (based on neuromorphic/robotics design kits), undergraduate and graduate students (Workshop courses, new classes/lectures at participants? universities and REU), and to established researchers (exposure to new areas in the field). The workshop will also continue to educate the Telluride community with public lectures on the latest developments/issues in the field.
Recruitment of minorities and women to the field will be continue by organizing lectures at various Universities, particularly HBCUs (Morgan State U., MD, Lincoln U., PA, Morehouse College, Atlanta, GA, and others). By sending presenters to institutions local to their home universities, minimal funding will be required and provide the most likely connections for future collaborations. The Institute for Neuromorphic Engineering, currently housed at the University of Maryland (College Park, MD), will arrange the logistics. The lectures and other teaching materials developed at the workshop will also be made available to all interested parties and posted on the INE website.
Lastly, the workshop will continue to develop the researchers and leaders for the emerging field of biologically-inspired systems, cognitive/learning systems, robotics and implantable electronics. Various agencies and governments have recognized that smart devices (such as interactive humanoid robots) that mimic living organisms will have great academic and commercial value in future.

EEC-0649069
Jerry L. Prince

This award for an REU Site at Johns Hopkins University (JHU) will engage undergraduate students who have career interests in engineering in exciting and challenging research experiences on computer integrated surgical (CIS) systems. These new surgical systems have the potential to reduce surgery and recovery time, reduce surgical errors, reduce patient suffering, improve efficiency, reduce costs, and most importantly, improve patient outcomes. Each student will be a part of a collegial research team, including a faculty project supervisor and a graduate student mentor. Participants will receive instruction on technical communication, oral presentation skills, and research ethics to aid in the completion of the required final research report and presentation.

Additional activities will include tours and trips to all Engineering Research Center (ERC) labs at Johns Hopkins, the opportunity to perform laparoscopic procedures at the JH Hospital Minimally Invasive Surgical Training Center, and industry tours to the JH Applied Physics Lab and local engineering industry companies.

Students from institutions nationwide, with a focus on women and underrepresented minorities and from a wide range of engineering disciplines including electrical, mechanical, biomedical, computer science, and physics will be recruited to participate in the program. This program will contribute to the development of new systems and methods enhancing the ability of surgeons to plan and execute minimally invasive surgical procedures, thus addressing a vital national need to improve the delivery of healthcare.

A. PROJECT SUMMARY
Vision - The goal of this proposal is to leverage and combine our education and research strengths in biomedically oriented engineering, robotics and clinical intervention to develop the North American School in Medical Robotics and Computer Integrated Interventional System (NAS MR/CIIS). MR/CIIS, as a field of study, requires unique cross-disciplinary training that encompasses elements of engineering, physical, medical, and biological sciences. We propose to develop an Winter School that will provide pre-doctoral students, post-docs and senior new-comers to MR/CIIS the tools to understand and contribute to the expanding field of computer integrated medical technologies and clinical interventions. We will provide tutorials, projects and research opportunities to conceptualize, design and develop computational sensing, information processing and robotics systems for medical interventions. A distinguishing feature of the NAS MR/CIIS is that it will provide exposure to techniques for designing systems that include both computational models/data and interventional technologies. From a broad perspective, the goal of the School is to foster the growth the field of MR/CIIS by providing a short course that marries engineering and clinical medicine.
Intellectual Merit ? The intellectual challenge in this proposal emerges from the near contradictory requirement to deliver more effective, more precise and highly specialized medical care to a large population, while containing its cost. In the past decade we have seen computerization and automation of various manufacturing sectors producing higher quality, and exquisitely precise products at much lower costs. MR/CIIS offer similar benefits to medicine ? i.e. the computerization and (semi-)automation, through robotics, of the performance of clinical tasks. This is not possible without significant technological education, research and innovation in computational imaging and sensing, information extraction and visualization, automated guidance and assistance, efficient human-computer interaction and accurate on-line assessment and feedback. Johns Hopkins University and its partners in the NSF ERC on Computer Integrated Surgical Systems and Technology (CISST) are ideally situated to make significant inroads on this problem because of strong engineering teams already working on many of these systems, premiere medical institutions and clinicians willing to collaborate and use these technologies. Hence, the NAS MR/CIIS will capitalize on existing expertise and facilities, to increase the number of individuals who can comfortably answer the technical questions for the above systems, while fully understanding their clinical implications.
Broader Impact ? The NAS MR/CIIS is by definition an educational program with far reaching implications. The goal of the School is to promote the growth of the field of MR/CIIS. Participants will have an accelerated path to research productivity. This acceleration is required to address the expected increase demand for medical interventions by the aging baby-boomers. Increasing the number of researchers in the field will increase technological innovation and deployment society-wide. The diversity of the participants in the program will benefit from our substantial experience in the recruiting and retention of women and minorities into our ERC CISST. We have relationships with LSAMPs and McNair programs in the area to recruit participants. We will also target HBCUs and other minority serving institutions by presenting MR/CIIS seminars at their institutions to generate interest in the field. We already have ties with some HBCUs, such as Morgan State and Howard Universities. We plan to also use our significant K ? 12 outreach contacts to invite two teachers to the NAS MR/CIIS staged in the US. To maximize the benefit of participation and to help transfer knowledge to their classrooms, each teacher will be paired with one of the leaders of the School as a mentee. We will develop a web portal, to be managed by the Computer Integrated Surgery Student Research Society (CISSRS), to disseminate MR/CIIS material, for discourse through Blogs, and to formally establish the community.
Key Words ? Winter School, Medical Robotics, Computer Integrated Surgery

This three-year REU Site program at Johns Hopkins University and Hospital (JHU/H) is multidisciplinary and offers undergraduate participants the ability to conduct research in a variety of fields and develop strong teamwork collaboration skills. During a ten-week summer session REU students will engage in exciting and challenging research projects in a wide range of engineering disciplines; e.g. electrical, mechanical and biomedical, computer science, and physics. Each participant will be matched with a current research project in the new Laboratory for Computational Sensing and Robotics (LCSR) and will be a part of a research team, including a faculty mentor and a graduate student mentor. These research projects relate to medical image registration and fusion, image enhancement and segmentation or the development of new robotic devices to support surgeons in the operating room or to aid patients with disabilities.

In order to enhance the directed research, many additional enrichment activities are included: 1) instruction on technical communication; 2) oral presentation skills; and 3) research ethics. Additional activities include tours and trips to other labs at JHU Hospital and the Applied Physics Laboratory, the opportunity to perform laparoscopic procedures at the JHU/H Minimally Invasive Surgical Training Center, and industry tours of local robotics, biotechnology and engineering companies. Students will also participate in social activities to foster team a sense of community within the undergraduate student group. The program will culminate with a program devoted to final presentations from REU participants with their PIs, graduate student mentors, lab mates, and parents all present. An award for the best presentation will be presented by LCSR leadership.

Recruitment efforts will be targeted to potential participants from institutions nationwide, with a focus on women and under-represented minorities, and from a wide range of engineering disciplines including electrical, mechanical, biomedical, computer science, mathematics, and physics. By recruiting from and partnering with LSAMP, McNair, SWE, SHPE, and other similar programs, this program will aid in the development of a pipeline of qualified, diverse practitioners who will contribute to the workforce in the area of STEM, particularly in the multi- and inter-disciplinary subjects encountered in the computational sensing, medical robotics, prosthetics, and computer integrated medical intervention areas.

Project Abstract: 1057644
The human hand, wrist and arm make up one of the most complex portions of the human body. Using our arms and hands, humans are able to perform extremely complex functions, ranging from the delicate and dexterous tasks involved in artistic design, through dynamic ones involved in playing musical instruments, to forceful ones involved in sports and labor. Scientific studies demonstrate that, even without seeing our hands, a person can effortlessly recognize hundreds of objects with his hands. Unfortunately, none of the arm and hand prosthetics that have been developed to-date are remotely capable of providing touch sensation that approaches that of the natural limbs. Yet, it is known that the touch sensation is indispensable for humans to effectively manipulate and explore objects. So it is a challenge is to assimilate amputees into society and provide them the tools to contribute to the workforce unless they are provided prosthetics limbs that move by thought, as well as feel what the prosthetic hand touches. This EAGER proposal specifically aims to improve our scientific knowledge of how touch is represented and learned by the brain, develop electronic systems that can be implanted to communicate touch directly to the brain, and to test the effectiveness of providing the sensation of touch to a monkey by circumventing its arm and communicating directly to the brain. If successful, this high-risk/high-pay-off project could make Luke Skywalker's replacement arm in Star Wars: The Empire Strikes Back a reality.
This project will develop new transdisciplinary knowledge involving neuroscience and engineering. The goal is to record and stimulate directly from the parts of the brain where the sense of touch is normally represented. Current research shows that normal perception of touch is provided by the activity of large groups of brain cells (i.e. neurons). The Investigators will study the possibility of using electrical stimulation to restore the sense of touch to amputees in the same way that cochlear implants restore hearing to the deaf or visual implants the sense of vision to the blind. They plan to exploit the natural representations of the brain and to stimulate, using new electronic circuitry, large groups of neurons that represent movement and from in the animals brain. Ultimately this research will lead to an understanding of how to recreate the feel of objects.
Throughout this work, the investigators will train students to have unique neuroscience, biomedical, and engineering skills, a combination of which is invaluable to the modern high-tech health related workforce. They plan to train both undergraduate and graduate students and expose K-12 students who regularly rotate through their laboratories to the research.

This INSPIRE award is partially funded by the Science of Learning Centers Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Perception, Action, and Cognition Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Energy, Power, and Adaptive Systems Program in the Division of Electrical Communication and Cyber Systems in the Directorate of Engineering; and the Applied Mathematics and Mathematical Biology Program in the Division of Mathematical Sciences in the Directorate for Mathematical and Physical Sciences. This research project draws on knowledge from many disciplines (neuroscience, cognitive science, computational science, mathematics and engineering) to create cognitive systems capable of interpreting observed, complex human movements and actions. New design methodologies will be developed for the integration of sensory modalities (vision, audition, touch) and their support of higher cognitive function (language, reasoning). In contrast to existing approaches which tend to be assemblies of modular components each solving its task in isolation, this team takes a novel approach called Active Cognition which has the following features: 1) Instead of modeling the different perceptual processes (vision, audition, and haptics), cognition, and motor control in isolation, the modules are integrated and capabilities co-developed in the tradition of dynamical systems theory to obtain a reasoning system where "the whole is greater than the sum of its parts"; 2) instead of segregating the low level processing of signals from the processing of higher level symbolic information, they will interact in a continuous dialogue, such that high level knowledge will leverage perception; and 3) instead of separating physical embodiment from algorithmic considerations, biologically inspired real-time hardware will be developed that implements complex functions by integrating signals and symbols. The project is organized in two working groups. The first group will develop a cognitive robot that can recognize complex human activities using visual and auditory signals captured by biological-inspired hardware. The second group will study attention in humans by measuring human response to audition and vision through EEG and MEG, and subsequently implementing the findings in robots. A yearly three-week, hands-on workshop will educate students, serve as testing ground for the team's ideas, and stimulate new collaborations. This workshop will also engage the involvement of the interdisciplinary research community that has formed around the goal of building biologically inspired cognitive systems.

Success in integrating different components of a cognitive system (hardware, sensors, and software) has the potential to catalyze a new industry of biologically-inspired cognitive systems, including household and service robots, and systems for intelligent transportation and smart manufacturing. In addition, this interdisciplinary project will play a significant role in building capacity for a new emphasis area in engineering and training of cognitive systems engineers who need combined expertise in computer science, electrical engineering and cognitive neuroscience.

Image guidance is a key enabling technology for millions of interventional procedures annually in medicine. Biopsies, ablative procedures, energy-based therapies, as well as minimally invasive and open procedures all rely on real-time feedback from imaging in order to be performed safely and effectively. In current practice, this feedback is predominantly provided by ultrasound (US) as it offers safe, real-time imaging for low cost. However, ultrasound often has poor image quality and is difficult to manage, particularly for interventions requiring high precision. These issues have led many research groups and several companies to investigate methods for enhanced image guidance.

The vision of this proposal is to advance ultrasound-guided cancer therapy with an innovative, low-cost, and self-contained navigation device which is being developed by Clear Guide Medical LLC, a small business spun out of the CISST ERC. The proposed device will enable the system to perform dynamic registration to a preoperative diagnostic image. The result will be a "Body GPS" system that is able to localize the probe in "body coordinates." This platform will then be further developed to provided targeting information via real-time, on patient targeting display.

The principle objectives are: 1) to develop algorithms to calibrate the elements of the system; 2) to provide surface representations of the patient via structured light stereo; 3) to develop projection modes for on-patient targeting display; and 4) to evaluate the resulting projection methods in a simulated biopsy environment. Objectives 1 and 2 will be performed in collaboration with the CISST ERC; while objectives 3 and 4 will be led by Clear Guide Medical.

One of the core research thrusts of the CISST ERC is the development of new technologies for percutaneous therapy (that is medical procedures where access to inner organs or other tissue is done via needle-puncture of the skin, rather than by using an "open" approach where inner organs or tissue are exposed). To create a viable device, it will be necessary to combine advanced methods in computer vision and ultrasound imaging with new methods for on-patient targeting display. The project team is led by PI Stolka, the head of engineering for Clear Guide Medical and an expert on ultrasound systems, and ERC PI Etienne-Cummings, and expert on compact computational imaging system. They are assisted by co-PI Boctor, an expert in ultrasound imaging and image guidance and co-PI Hager, CEO of Clear Guide Medical and an expert in computer vision.

BROADER SIGNIFICANCE OF THE PROJECT:

This project will provide undergraduates STEM research experience at the interface of engineering and medicine, aiming to help medical diagnosis and procedures, development of prosthetics, and contribute to technologies that are likely to become a part of the "future of medicine." This program addresses a vital national need to improve the delivery of healthcare by developing new tests and medical devices that enhance the ability of doctors to plan and perform procedures. By recruiting from and partnering with minority student societies and programs, minority-serving institutions and community colleges, the investigators will help develop a pipeline of qualified, diverse individuals who will contribute to the STEM workforce, particularly in the multi- and interdisciplinary subjects encountered in biomedical research, healthcare delivery, and basic biological and life sciences. The participants will be well trained in communications and research ethics, which are essential for success in today's biotechnology and bioscience work and market place.

PROJECT DESCRIPTION:

During a ten-week summer session, undergraduate participants from institutions nationwide will engage in exciting and challenging research projects in a wide range of engineering disciplines, e.g. electrical, mechanical and biomedical, computer science, and physics. Each participant will be matched with a current research project in the Laboratory for Computational Sensing and Robotics (LCSR) and will be a part of a collegial research team, including a faculty project supervisor and a graduate student mentor. These research projects relate to medical image registration and fusion, image enhancement and segmentation or the development of new robotic devices to support surgeons in the operating room or to aid patients with disabilities. Participants will receive instruction on technical communication, oral presentation skills, and research ethics and will deliver a final research report and presentation. Additional activities will include tours and trips to other labs at JHU Hospital (JHU/H) and the Applied Physics Laboratory, the opportunity to perform laparoscopic procedures at the JHU/H Minimally Invasive Surgical Training Center, and industry tours of local robotics, biotechnology and engineering companies.

The program is multidisciplinary and offers each participant the ability to conduct research in a variety of fields and develop strong teamwork collaboration skills. Each faculty mentor provides a project description for projects that may be created specifically for the program or designed to carry out a facet of an on-going research. Because the Laboratory for Computational Sensing and Robotics has close ties with the Johns Hopkins Medical Institutions, participants can experience the cutting-edge research that is designed to aid medical diagnosis, interventions, and prosthesis, and contribute to technologies that are likely to become a part of the "future of medicine." The investigators plan to leverage on-going research activities and training and mentoring experience, and aim this REU in CS&MR program at broader topics that include more biological inspired and biologically targeted computational sensing, imaging, and robotics systems. An external assessment expert will conduct annual formative and summative evaluations.

The motivation for this biologically-inspired approach is to design systems that perceive and act in cluttered and noisy scenes that they have never experienced. This stands in contrast with the state of the art in computational engineering systems that need to be re-trained each time they confront an unanticipated environment. The main reason is that current approaches to perception address specific problems in isolation and do not consider that the primary role of perception is to support systems with bodies in action. As a result, they are constrained to the situations for which they were trained and cannot react to changing tasks and scenes. By focusing on cognition primitives rather than specific applications, the work is expected to greatly advance the state of the art of machine perception and lead to the development of systems that can robustly and on-line adapt to new environments, react to novel situations and learn new contexts. To do so, novel theoretical formulations of perception and action and high-speed, low-power, hardware implementations with on-line learning capabilities will be studied while assimilating new insights from the neurosciences. Consequently, this work will network neuroscience, cognitive science, applied mathematics, computer science and engineering so as to lower one of the few remaining barriers that keeps interactive robots in the realm of science fiction. Beyond the scholarly contribution, the work is expected to provide know-how for the design of systems with adaptive perception in a modular fashion with reusable components. Such systems have applications in computational vision and auditory perception problems and can advance the industry of cognitive biologically-inspired robotics and assistive devices.

This proposal sets forward novel ideas in the design of intelligent perceptual systems and the development of synthetic intelligence. Just about any task which an intelligent system solves involves the interplay of four basic processes that are devoted to: (a) context, (b) attention, (c) segmentation and (d) categorization. The members of the proposed network will study these canonical cognitive primitives by combining neural modeling with neural and behavioral experiments, theoretical and computational modeling and implementation in robotics. The findings of theoretical insights will then be adapted to satisfy the demands of realistic behavior, and to develop technological solutions for applications of robust and invariant perception and action. The proposed collaborative network will consist of a small science and engineering research team to directly address the questions in robust adaptive perception and action. It will then direct personnel, and inject results and pedagogical content to a Summer Workshop that aims to include a global network of researchers.

The visual brain infers a three-dimensional world from twodimensional images and organizes the visual information in terms of objects in three-dimensional space, representing even objects that are partially occluded and appear fragmented in the retinal image. This organization is the basis for attentive selection, action planning and object recognition. A combination of experimental and theoretical studies together with model implementations in neuromorphic hardware will be used to elucidate the interface between visual feature representations and attentive cognitive processes. Previous findings on the neural coding of figure-ground structure can be understood in terms of grouping mechanisms that structure the incoming sensory information as proto-objects (objects as defined by the system at this stage). The grouping mechanisms also provide handles for top-down mechanisms to address and select object-related information. The proposed work will explain how neuronal circuitry organizes spatially disconnected visual features into perceptual objects. How is this implemented neurally to lead to a coherent representation? Detailed computational models of the underlying circuitry will be developed, both as standard numerical simulations and in fast, neuromorphic hardware, and then tested by multiple single-cell recordings in awake non-human primates. Specifically, while prior studies examined spike time correlations indiscriminately in all neurons, our recent studies differentiated neurons according to their role in the grouping circuits. The grouping hypothesis predicts elevated synchrony only in pairs of neurons that belong to the same grouping circuit, but not in other pairs. These model predictions were confirmed in a recent study which showed that spike-spike correlation functions are in qualitative agreement with the idea that perceptual grouping is implemented by feedback from populations of dedicated grouping cells. Quantitative understanding requires the development of explicit spiking models, which is one of the main foci of this proposal. Models will be implemented on neuromorphic spiking hardware since the complexity of the cortical circuitry makes realistic model simulations on CPU/GPU system impossible. Predictions of integrate-and-fire type models of this circuitry will be compared with rate and synchrony observed in our recordings and deviates used to fine-tune the models. We will pursue the educational and broader impacts aims on five fronts. 1) Students will be crosstrained and mentored in biological, mathematical and engineering sciences, which will lead to graduates with unique skill sets. 2) We will contribute to the development of the nascent neuromorphic engineering field, providing new research problems that can benefit from the crosstraining and collaboration. We plan to participate in the NSF sponsored Telluride Neuromorphic Cognition Engineering and Capo Caccia Neuromorphic Cognitive Engineering Workshops for this purpose. 3)We will provide an opportunity for undergraduate students to participate in the research as part of our Site REU (managed by one of the PIs). They are trained in communications, research ethics and project management, which are crucial for success in todays biotechnology and bioscience work and market place. 4) We currently host students from local high-schools who conduct STEM research practicum rotations in our labs. This project will provide a perfect venue for the rotators to get exposed and mentored on multi-disciplinary research problems. We will use a tiered mentoring structure, where undergraduates mentor K-12 rotators, graduate students mentor undergraduates, and faculty members mentor all participants. 5) Our student recruitment plans will build on our current partnerships with MARC, LSAMP, McNair, SWE, SHPE and other similar programs and minority-serving institutions and local community colleges, to help develop a pipeline of qualified, diverse individuals who will contribute to the workforce in the area of STEM.

The objective of this Research Coordination Network (RCN) is to advance understanding of how biological systems learn complex symbolic signals, and create artificial systems with similar capabilities. By defining a common framework to describe these signals, and their variability across space and time, the RCN will develop methods and tools applicable to a wide range of domains, including language, music, action, perception, and navigation. The RCN will build upon research in Neuromorphic Engineering and its development of bio-inspired, low-power computing platforms, sensors, and signal processing. Using these tools, the RCN will focus on high-level cognitive functions, to create complex, bio-inspired systems that learn through engagement in tasks. The network will bring together neuroscience, cognitive science, applied mathematics, computer science, and engineering, with emphasis on machine learning and artificial intelligence. Network members will participate in a yearly three-week, hands-on workshop, that will develop and test new tools and ideas, stimulate new collaborations, and educate students on unique interdisciplinary skills.

The RCN will facilitate interactions and collaborative projects among participating researchers employing a wide range of paradigms that specifically deal with three thrusts: the role of neural plasticity for learning symbolic systems; the adaptive mechanisms underlying the learning of sensory-motor tasks; and transitioning to real-world applications such as automatic speech and dynamic scene understanding, neuromorphic hardware implementations, cognitive computational algorithms, and databases acquisition. Specific examples of such diverse projects include brain process models that assess learning and expertise; algorithms, based on physiological or abstract events, that process input from neuromorphic hardware; and development of software and neuromorphic hardware for signal interpretation and action execution.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

The broader impact/commercial potential of this Partnerships for Innovation – Research Partnerships (PFI-RP) project is to reduce the burden of heart failure across the healthcare system. Heart failure is the most common cause for hospital admission and readmission in the US (> one million patients annually). Many heart failure readmissions are thought to be preventable. To reduce readmissions, healthcare organizations are penalized for high rates of 30-day hospital readmission. Excess readmission penalties were ~$560 million across all hospitals in the US in 2020, with ~2,500 hospitals incurring penalties. Customer discovery interviews, conducted with numerous stakeholders of healthcare organizations, suggested a strong interest in a Clinical Decision Support tool to identify patients diagnosed with heart failure and who are at high risk of 30-day hospital readmission. By accurately stratifying risk for 30-day hospital readmission, this software tool empowers clinicians involved in discharge planning to make more informed decisions about the timing of hospital discharge and the efficient use of post-discharge follow-up services, including allocation of remote monitoring hardware. This solution can improve patient outcomes while reducing costs associated with avoidable hospitalizations and the corresponding penalties for hospitals.

The proposed project focuses on the development and commercialization of a novel, machine learning-based clinical decision support software tool to predict 30-day readmissions for hospitalized patients diagnosed with heart failure. The proposed technology includes higher frequency physiologic data in the predictive algorithm and the software tool will be capable of processing this data in combination with clinical variables with low latency (<3 seconds) to quantify each patient’s risk of 30-day hospital readmission at the point of care. This technology has the potential to assist with the identification and management of high-risk patients diagnosed with heart failure and improve the quality of care for this vulnerable patient population.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Artificial intelligence is becoming ubiquitous in modern life. To build systems under the current paradigm, large amounts of energy are required for computing and sensing. This causes environmental problems, pollution, and challenges for small-sized systems, as well as privacy issues. The field of neuromorphic science and technology offers an alternative by seeking to understand principles of biological brains and build on their basis artificial systems using low-power hardware and software solutions. While its advantages have been demonstrated, further advances are necessary and will require common computational tools and principled experimental approaches. This AccelNet project, NeuroPacNet, links international experts in neuromorphic engineering with computational neuroscientists, roboticists, control theorists, and researchers of perception from seven global networks to set the foundations for building systems that can robustly process real-world signals in time and adapt to changes. This network of networks will facilitate the development of new methods and approaches for intelligent system design and prepare the next generation of leaders in neuromorphic science and technology. As different industries adopt neuromorphic hardware, society will have access to new applications, such as in computing on cell phones, neuroprostheses, intelligent hearing aids, and smart sensory systems with predictive capabilities.

NeuroPacNet will advance computational research on modeling the integration of perception, action, and cognition. The network of network will coordinate across those research thrusts and develop new approaches grounded in theoretical neuroscience for sensorimotor control, motor learning, event-based computations, and learning in spiking neural networks. NeuroPacNet will also include robotics research in the areas of drone navigation and human activity understanding for humanoids and will address social and ethical issues in humanoid robotics. The network of networks will use innovative hardware design and mixed signals computational systems to address computation for emerging and unconventional technologies. International collaboration and knowledge exchange will include an immersive research exchange program providing scholarships to students and postdoctoral researchers, an annual workshop to discuss common issues and concerns in a stimulating environment and to engage in hands-on projects, meetings to define challenges, opportunities, and actions to accelerate progress, and competitions with two challenges to be solved by teams of researchers and students. An interactive project website will become a portal for archived webinar talks, tools, and data.

The Accelerating Research through International Network-to-Network Collaborations (AccelNet) program is designed to accelerate the process of scientific discovery and prepare the next generation of U.S. researchers for multiteam international collaborations. The AccelNet program supports strategic linkages among U.S. research networks and complementary networks abroad that will leverage research and educational resources to tackle grand scientific challenges that require significant coordinated international efforts.

Co-funding for this project is provided by the Directorate for Social, Behavioral, and Economic Sciences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.