Ralph Etienne-Cummings - US grants

ABSTRACT

Although digital signal representation has become almost universal, there are still many areas where an analog representation is required to interface with an analog world or to meet various other objectives such as power dissipation, frequency, or cost. In these domains analog signal representation is essential for many input modalities such as instrumentation, sensor interfaces, and communications. Likewise, there are related output applications, such as biomedical actuation and industrial control. In addition, the needs of wireless and fiber-optical communication have reinvigorated analog design. However, there are serious problems concerning how to keep these analog components on a reasonable scaling curve as Moore's law continues unabated in the digital domain, and in integrating analog representations into large, complex digital systems ("system on a chip").

The purpose of this proposal is to study a new approach to representing analog signals that we believe will integrate more cleanly into these deep submicron, single-chip systems. Today analog signals are almost exclusively represented by current or voltage quantities. Our proposal is to borrow a page from neuroscience and to use the Inter-Pulse-Interval (IPI) between single-bit, asynchronous pulses to represent analog quantities. We are proposing to develop a mixed-mode analog/digital cell library and design methodology based on IPI representation and the associated computation elements, and to engineer a case study to illustrate the outcomes.

As we move to deep submicron and then on to the nanometer/molecular devices, the problems that digital encounters with scaling, such as threshold inconsistency, subthreshold currents, hot-electron effects, doping variability, substrate coupling, and transmission line and complex cross-talk effects, are even more serious for analog circuitry. IPI representations will provide significantly better immunity to these effects, as well as to the more traditional process, temperature, and reference voltage variations. For most applications, pulse based analog systems will require less power. There are numerous advantages to using pulses or pulses to communicate. They are significantly more immune to noise. An approximate analogy would be that of AM versus FM radio signal representation. The outcomes of the proposed research are

Create a library of basic communication, computation (arithmetic and logic) and conversion building blocks;
Design, implementation and testing of a case study using the derived building blocks and methodology;

Document an IPI-based design methodology

Fellows will rotate through an internship with several distinct K-12 schools that serve disadvantaged children. Master Teachers who have been involved in research at JHU or who are currently participants in a Native American Math Science Partnership will work with the Fellows: to develop pedagogical skills to teach children with different learning styles; to enhance the content knowledge of district teachers; and to facilitate the creation of standards aligned content based on cutting-edge research. Each project will focus on topics from Environmental Engineering and Geography (EEG), including geology, hydrology, ecology, geomorphology, environmental chemistry, human factors (relations between human activities and environmental change). The Fellows will relate each project: (1) to standards, such as the American Association for the Advancement of Science.s Project 2061, Flow of Matter in Ecosystems;. (2) to compelling social issues as global climate change, and preservation of ecosystems; and (3) to the utilization of materials by students with physical limitations.

The intellectual merit lies in the potential to advance our understanding of how formal university-K-12 partnerships can improve teaching and learning by delivering challenging and relevant science, technology, engineering and mathematics (STEM) content to traditionally disadvantaged K- 12 students with a wide range of learning styles. A team of eminent scientists, future STEM faculty, leading K-12 teachers, and education experts will work as teams to use engaging EEG content and pedagogy to overcome barriers such as: students. unstable learning environments, curriculums that are in transition due to educational policy shifts, the lack of accommodations for students with physical disabilities, and the scarcity of resources.

Broader impact will be achieved through the development of sustainable K-12 curriculum and laboratory modules deployed by teachers prepared to deliver advanced, multi-disciplinary, and multi-contextual material spanning the STEM disciplines. An independent evaluator will gather information and report the findings on the feasibility of using EEG activities and instructional materials to improve the academic achievement of disadvantaged students with diverse needs.

The Telluride Workshop on Neuromorphic Cognition Engineering
Neuromorphic engineers design and fabricate artificial neural systems whose detailed architecture, design, and computational principles are based on those of biological nervous systems. Over the past 12 years, this research community has focused on the understanding of low-level sensory processing and systems infrastructure; efforts are now expanding to apply this knowledge and infrastructure to addressing higher-level problems in perception, cognition, and learning.
The annual three-week intensive Workshop (held in Telluride, Colorado) consists of background lectures (from leading researchers in biological, cognitive, computational, engineering and learning sciences), practical tutorials (from state-of-the-art practitioners), hands-on projects (involving established researchers and newcomers/students), and special interest discussion groups (proposed by the workshop participants). For researchers in this community, this is the premier workshop for training students, initiating collaborations, and in-depth discussions on scientific issues.
In this workshop and through the Institute for Neuromorphic Engineering (INE), the mission is to promote interaction between senior and junior researchers; to educate new members of the community; to introduce new enabling fields and applications to the community; to promote on-going collaborative activities emerging from the Workshop, and to promote a self-sustaining research field.
Specific Goals for the period of 2007-2012: While there is no question that the Workshop has been very successful in its mission, three new challenges have been identified for the Workshop: 1) with a rapidly expanding community in both the U.S. and Europe, the Workshop experience needs to reach more people without increasing the size of the Workshop, 2) as larger and more challenging projects are tackled, more opportunities for group interactions are needed throughout the year, and 3) as more complex questions are asked at the system-level, more voices from cognitive neuroscience are needed.
To meet these new challenges, a new version of the Workshop is envisioned with: 1) an expanded theme to focus on Perception, Cognition, and Learning, 2) an expanded constituency, educational mandate and research focus to incorporate members of the NSF Science of Learning Centers (SLC), 3) to create a two-part Workshop series (to allow yearlong collaborations and deeper investigation into large scale projects), one held in the U.S. and funded by U.S. resources and the other held in Europe and supported by European resources and 4) a modified Workshop schedule to emphasize training at the beginning of the workshop to provide a needed focus on education for both beginners and experts alike. The infusion of new researchers (from the SLCs) that focus on learning at multiple scales (from synapses to classroom) will provide the needed knowledge, new collaborations, and new perspectives to move the community towards cognitive-level neuromorphic systems.
Broader impact of the Workshop to the public: The Telluride Neuromorphic Cognition Engineering Workshop will continue its tradition of public interaction. In particular, there will be a continuation of the educational program for K-12 students (based on neuromorphic/robotics design kits), undergraduate and graduate students (Workshop courses, new classes/lectures at participants? universities and REU), and to established researchers (exposure to new areas in the field). The workshop will also continue to educate the Telluride community with public lectures on the latest developments/issues in the field.
Recruitment of minorities and women to the field will be continue by organizing lectures at various Universities, particularly HBCUs (Morgan State U., MD, Lincoln U., PA, Morehouse College, Atlanta, GA, and others). By sending presenters to institutions local to their home universities, minimal funding will be required and provide the most likely connections for future collaborations. The Institute for Neuromorphic Engineering, currently housed at the University of Maryland (College Park, MD), will arrange the logistics. The lectures and other teaching materials developed at the workshop will also be made available to all interested parties and posted on the INE website.
Lastly, the workshop will continue to develop the researchers and leaders for the emerging field of biologically-inspired systems, cognitive/learning systems, robotics and implantable electronics. Various agencies and governments have recognized that smart devices (such as interactive humanoid robots) that mimic living organisms will have great academic and commercial value in future.

This three-year REU Site program at Johns Hopkins University and Hospital (JHU/H) is multidisciplinary and offers undergraduate participants the ability to conduct research in a variety of fields and develop strong teamwork collaboration skills. During a ten-week summer session REU students will engage in exciting and challenging research projects in a wide range of engineering disciplines; e.g. electrical, mechanical and biomedical, computer science, and physics. Each participant will be matched with a current research project in the new Laboratory for Computational Sensing and Robotics (LCSR) and will be a part of a research team, including a faculty mentor and a graduate student mentor. These research projects relate to medical image registration and fusion, image enhancement and segmentation or the development of new robotic devices to support surgeons in the operating room or to aid patients with disabilities.

In order to enhance the directed research, many additional enrichment activities are included: 1) instruction on technical communication; 2) oral presentation skills; and 3) research ethics. Additional activities include tours and trips to other labs at JHU Hospital and the Applied Physics Laboratory, the opportunity to perform laparoscopic procedures at the JHU/H Minimally Invasive Surgical Training Center, and industry tours of local robotics, biotechnology and engineering companies. Students will also participate in social activities to foster team a sense of community within the undergraduate student group. The program will culminate with a program devoted to final presentations from REU participants with their PIs, graduate student mentors, lab mates, and parents all present. An award for the best presentation will be presented by LCSR leadership.

Recruitment efforts will be targeted to potential participants from institutions nationwide, with a focus on women and under-represented minorities, and from a wide range of engineering disciplines including electrical, mechanical, biomedical, computer science, mathematics, and physics. By recruiting from and partnering with LSAMP, McNair, SWE, SHPE, and other similar programs, this program will aid in the development of a pipeline of qualified, diverse practitioners who will contribute to the workforce in the area of STEM, particularly in the multi- and inter-disciplinary subjects encountered in the computational sensing, medical robotics, prosthetics, and computer integrated medical intervention areas.

This INSPIRE award is partially funded by the Science of Learning Centers Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Perception, Action, and Cognition Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Energy, Power, and Adaptive Systems Program in the Division of Electrical Communication and Cyber Systems in the Directorate of Engineering; and the Applied Mathematics and Mathematical Biology Program in the Division of Mathematical Sciences in the Directorate for Mathematical and Physical Sciences. This research project draws on knowledge from many disciplines (neuroscience, cognitive science, computational science, mathematics and engineering) to create cognitive systems capable of interpreting observed, complex human movements and actions. New design methodologies will be developed for the integration of sensory modalities (vision, audition, touch) and their support of higher cognitive function (language, reasoning). In contrast to existing approaches which tend to be assemblies of modular components each solving its task in isolation, this team takes a novel approach called Active Cognition which has the following features: 1) Instead of modeling the different perceptual processes (vision, audition, and haptics), cognition, and motor control in isolation, the modules are integrated and capabilities co-developed in the tradition of dynamical systems theory to obtain a reasoning system where "the whole is greater than the sum of its parts"; 2) instead of segregating the low level processing of signals from the processing of higher level symbolic information, they will interact in a continuous dialogue, such that high level knowledge will leverage perception; and 3) instead of separating physical embodiment from algorithmic considerations, biologically inspired real-time hardware will be developed that implements complex functions by integrating signals and symbols. The project is organized in two working groups. The first group will develop a cognitive robot that can recognize complex human activities using visual and auditory signals captured by biological-inspired hardware. The second group will study attention in humans by measuring human response to audition and vision through EEG and MEG, and subsequently implementing the findings in robots. A yearly three-week, hands-on workshop will educate students, serve as testing ground for the team's ideas, and stimulate new collaborations. This workshop will also engage the involvement of the interdisciplinary research community that has formed around the goal of building biologically inspired cognitive systems.

Success in integrating different components of a cognitive system (hardware, sensors, and software) has the potential to catalyze a new industry of biologically-inspired cognitive systems, including household and service robots, and systems for intelligent transportation and smart manufacturing. In addition, this interdisciplinary project will play a significant role in building capacity for a new emphasis area in engineering and training of cognitive systems engineers who need combined expertise in computer science, electrical engineering and cognitive neuroscience.

BROADER SIGNIFICANCE OF THE PROJECT:

This project will provide undergraduates STEM research experience at the interface of engineering and medicine, aiming to help medical diagnosis and procedures, development of prosthetics, and contribute to technologies that are likely to become a part of the "future of medicine." This program addresses a vital national need to improve the delivery of healthcare by developing new tests and medical devices that enhance the ability of doctors to plan and perform procedures. By recruiting from and partnering with minority student societies and programs, minority-serving institutions and community colleges, the investigators will help develop a pipeline of qualified, diverse individuals who will contribute to the STEM workforce, particularly in the multi- and interdisciplinary subjects encountered in biomedical research, healthcare delivery, and basic biological and life sciences. The participants will be well trained in communications and research ethics, which are essential for success in today's biotechnology and bioscience work and market place.

PROJECT DESCRIPTION:

During a ten-week summer session, undergraduate participants from institutions nationwide will engage in exciting and challenging research projects in a wide range of engineering disciplines, e.g. electrical, mechanical and biomedical, computer science, and physics. Each participant will be matched with a current research project in the Laboratory for Computational Sensing and Robotics (LCSR) and will be a part of a collegial research team, including a faculty project supervisor and a graduate student mentor. These research projects relate to medical image registration and fusion, image enhancement and segmentation or the development of new robotic devices to support surgeons in the operating room or to aid patients with disabilities. Participants will receive instruction on technical communication, oral presentation skills, and research ethics and will deliver a final research report and presentation. Additional activities will include tours and trips to other labs at JHU Hospital (JHU/H) and the Applied Physics Laboratory, the opportunity to perform laparoscopic procedures at the JHU/H Minimally Invasive Surgical Training Center, and industry tours of local robotics, biotechnology and engineering companies.

The program is multidisciplinary and offers each participant the ability to conduct research in a variety of fields and develop strong teamwork collaboration skills. Each faculty mentor provides a project description for projects that may be created specifically for the program or designed to carry out a facet of an on-going research. Because the Laboratory for Computational Sensing and Robotics has close ties with the Johns Hopkins Medical Institutions, participants can experience the cutting-edge research that is designed to aid medical diagnosis, interventions, and prosthesis, and contribute to technologies that are likely to become a part of the "future of medicine." The investigators plan to leverage on-going research activities and training and mentoring experience, and aim this REU in CS&MR program at broader topics that include more biological inspired and biologically targeted computational sensing, imaging, and robotics systems. An external assessment expert will conduct annual formative and summative evaluations.

The visual brain infers a three-dimensional world from twodimensional images and organizes the visual information in terms of objects in three-dimensional space, representing even objects that are partially occluded and appear fragmented in the retinal image. This organization is the basis for attentive selection, action planning and object recognition. A combination of experimental and theoretical studies together with model implementations in neuromorphic hardware will be used to elucidate the interface between visual feature representations and attentive cognitive processes. Previous findings on the neural coding of figure-ground structure can be understood in terms of grouping mechanisms that structure the incoming sensory information as proto-objects (objects as defined by the system at this stage). The grouping mechanisms also provide handles for top-down mechanisms to address and select object-related information. The proposed work will explain how neuronal circuitry organizes spatially disconnected visual features into perceptual objects. How is this implemented neurally to lead to a coherent representation? Detailed computational models of the underlying circuitry will be developed, both as standard numerical simulations and in fast, neuromorphic hardware, and then tested by multiple single-cell recordings in awake non-human primates. Specifically, while prior studies examined spike time correlations indiscriminately in all neurons, our recent studies differentiated neurons according to their role in the grouping circuits. The grouping hypothesis predicts elevated synchrony only in pairs of neurons that belong to the same grouping circuit, but not in other pairs. These model predictions were confirmed in a recent study which showed that spike-spike correlation functions are in qualitative agreement with the idea that perceptual grouping is implemented by feedback from populations of dedicated grouping cells. Quantitative understanding requires the development of explicit spiking models, which is one of the main foci of this proposal. Models will be implemented on neuromorphic spiking hardware since the complexity of the cortical circuitry makes realistic model simulations on CPU/GPU system impossible. Predictions of integrate-and-fire type models of this circuitry will be compared with rate and synchrony observed in our recordings and deviates used to fine-tune the models. We will pursue the educational and broader impacts aims on five fronts. 1) Students will be crosstrained and mentored in biological, mathematical and engineering sciences, which will lead to graduates with unique skill sets. 2) We will contribute to the development of the nascent neuromorphic engineering field, providing new research problems that can benefit from the crosstraining and collaboration. We plan to participate in the NSF sponsored Telluride Neuromorphic Cognition Engineering and Capo Caccia Neuromorphic Cognitive Engineering Workshops for this purpose. 3)We will provide an opportunity for undergraduate students to participate in the research as part of our Site REU (managed by one of the PIs). They are trained in communications, research ethics and project management, which are crucial for success in todays biotechnology and bioscience work and market place. 4) We currently host students from local high-schools who conduct STEM research practicum rotations in our labs. This project will provide a perfect venue for the rotators to get exposed and mentored on multi-disciplinary research problems. We will use a tiered mentoring structure, where undergraduates mentor K-12 rotators, graduate students mentor undergraduates, and faculty members mentor all participants. 5) Our student recruitment plans will build on our current partnerships with MARC, LSAMP, McNair, SWE, SHPE and other similar programs and minority-serving institutions and local community colleges, to help develop a pipeline of qualified, diverse individuals who will contribute to the workforce in the area of STEM.

The broader impact/commercial potential of this Partnerships for Innovation – Research Partnerships (PFI-RP) project is to reduce the burden of heart failure across the healthcare system. Heart failure is the most common cause for hospital admission and readmission in the US (> one million patients annually). Many heart failure readmissions are thought to be preventable. To reduce readmissions, healthcare organizations are penalized for high rates of 30-day hospital readmission. Excess readmission penalties were ~$560 million across all hospitals in the US in 2020, with ~2,500 hospitals incurring penalties. Customer discovery interviews, conducted with numerous stakeholders of healthcare organizations, suggested a strong interest in a Clinical Decision Support tool to identify patients diagnosed with heart failure and who are at high risk of 30-day hospital readmission. By accurately stratifying risk for 30-day hospital readmission, this software tool empowers clinicians involved in discharge planning to make more informed decisions about the timing of hospital discharge and the efficient use of post-discharge follow-up services, including allocation of remote monitoring hardware. This solution can improve patient outcomes while reducing costs associated with avoidable hospitalizations and the corresponding penalties for hospitals.

The proposed project focuses on the development and commercialization of a novel, machine learning-based clinical decision support software tool to predict 30-day readmissions for hospitalized patients diagnosed with heart failure. The proposed technology includes higher frequency physiologic data in the predictive algorithm and the software tool will be capable of processing this data in combination with clinical variables with low latency (<3 seconds) to quantify each patient’s risk of 30-day hospital readmission at the point of care. This technology has the potential to assist with the identification and management of high-risk patients diagnosed with heart failure and improve the quality of care for this vulnerable patient population.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.