Roozbeh Jafari, Ph.D. - US grants

Providing an environment that offers both immersion and interaction is a tough research challenge. Ensuring a reasonable Quality of Experience (QoE) in using these environments installed in geographically distributed cities is even a tougher challenge. This project considers a collaborative, immersive, and interactive environment that not only supports 3D rendering of the participants? video but also other modalities such as Body Sensor Network (BSN) data that can offer highly precise data about a person?s physical movements (as well as physiological data). While creating this environment, one needs to consider the various bottlenecks that choke the data streams carrying the immersive and interactive information: reconstruction delay, ultra-high throughput needed, packet loss, and rendering delays.

The main aim of this project is to design and develop collaborative, multi-modal immersive environments with higher frame rates and frame quality by carrying out research tasks that can take advantage of information from other modalities and handle these bottlenecks.

In a typical tele-immersive environment, participants can see themselves in the locally rendered 3D view and see participants in the remote environments as well. Since the local rendering delays are much smaller, participants can see themselves earlier and in a more smooth fashion compared to the rendering of remote participants that suffers from communication delays and packet losses. This aspect of varying delays among the immersive participants can potentially cause problems during dynamic interactions and affect their QoE. Answers to questions such as what type of problems can be caused and how the participants handle them depend on the application domain of the immersive environments. To study the QoE and validate (with usability studies) the collaborative, immersive environment, a tele-rehabilitation application will be deployed in multiple cities: Berkeley, California; 2 sites in Dallas, Texas; and Urbana-Champaign, Illinois.

Intellectual Merits of this project are (i) The resource adaptation framework for streaming multi-source, multi-destination, multi-rate, multi-modal data incorporates supervisory hybrid control theory based fine-grained resource management, multi-modal coarse-grained management, and a multi-modal multicasting approach. (ii) Graphics Processing Unit (GPU)-based 3D reconstruction and compression algorithms. These algorithms facilitate reconstruction of 3D data points based on 3D camera array data and compress them at a faster pace than their CPU-based counterparts. (iii) GPU-based rendering algorithm of 3D data on the receiver side. This algorithm will handle potential data loss in 3D camera data streams using skeletal information from BSN data streams. (iv) Identification and measurement of Quality of Experience (QoE) metrics and using those metrics to derive Quality of Service (QoS) parameters. The derived QoS parameters will then help the resource adaptation framework to modify its decisions at run-time. This project aims to have transformative aspects in the new set of algorithms that exploits multi-modality while incorporating a feedback based on Quality of Experience for functions such as streaming, 3D reconstruction, and rendering.

Broader Impacts: This project promises significant impact in the fields of education and pervasive health care by providing augmented abilities to carry out intricate programs such as tele-rehabilitation with increased correctness and flexibility. This can also lead to improved productivity in the society considering the ability of health-care professionals to potentially handle a larger population (in remote places) as well as considering the possibility of the affected persons to become independent and productive faster. The project also ensures the results from the proposed research will be incorporated into the courses being taught. 3 women PhD students and 6 under-graduate students (2 are minority students) already working with the investigators of this project. Serious efforts will be undertaken to continue their involvement in this project. Apart from refereed conference and journal publications, the developed software, collected data, and research results will be shared with other researchers through a dedicated website (after ensuring satisfaction of HIPAA regulations).

Wearable computers are gaining significant attention due to their capability to enable a wide variety of new applications in domains such as wellness and health care. Despite their tremendous potential to impact our lives, wearable health monitoring systems face a number of hurdles before becoming a reality. The enabling processors and architectures demand a large amount of energy, requiring sizable batteries. This creates challenges for further miniaturization of the wearable units. This EAGER award is pursuing preliminary research in tiered, model based signal processing that can exploit pre-determined signal templates to enable extreme power optimization. In this approach, signal processing can be performed at several levels, where in each level, only the hardware for a specific template is active. If the template of interest is present, the next level of signal processing will be activated, otherwise hardware components corresponding to the next and the remaining levels will remain inactive. This approach, however, highly depends on the effectiveness of templates. In monitoring human movements, if templates do not accurately represent the physical activity of interest, the system will not exhibit acceptable accuracy. The goal is to develop effective techniques and methodologies to ensure templates adapt to remain valid throughout the operation of the system, accurately representing the corresponding physical movements.

The research focuses on speed-insensitive template matching architectures that can reduce the effects of movement variations on signal processing. Timing models for movements and user activity profiles are exploited to monitor the correctness of the signal processing, and tunable parameters decrease or increase the sensitivity of the signal processing. For example, if the user is expected to perform sit to stand at least once every two hours in the day time, and the tiered signal processing has not detected the movement in the past few hours, the sensitivity will be increased, or user interaction and template retraining can be initiated. When performing a movement that has been determined to be of interest, the user can initiate (re)training if the system does not recognize the movement. Effective template generation and on-line retraining are expected to open opportunities to individualize systems and signal processing and to reduce the complexity of storage and processing architectures. This research is expected to provide the groundwork for ongoing design and development of practical ultra low power signal processing architectures, reduce costs of computing platforms for medical sensing, and to enable future progress in areas such as gait and balance monitoring for fall prevention, and in-home movement monitoring for Parkinson?s disease.

DESCRIPTION (provided by applicant): Falls among the elderly are a considerable health concern and the leading cause of injury death. Fear of falling can be equally devastating leading to a loss of confidence, restriction of physical activities, and social isolation. Lateral sway while walking, in addition to other gait parameters, have been shown to be associated with a history of falls among the elderly. We propose the development of a wearable sensor system capable of identifying sway and gait parameters associated with fall likelihood and of providing corrective feedback. The system will characterize several parameter of sway and gait during standing on one leg and walking. The biofeedback system, consisting of auditory and vibratory feedback modules, will provide feedback to reduce the 'unsafe'sway. During the proposed study, the robustness and sensitivity of measurements will be established, and the effectiveness of the biofeedback system will be evaluated. The system will be capable of continuous monitoring and will be discrete so that it can be worn continuously, and hence increase confidence and the quality of life among elderly susceptible to falls. Lastly, in future, the system can be used in the clinics as a tool for evaluating the risks of falls, and training users to better maintain their balance. PUBLIC HEALTH RELEVANCE: Evaluating fall risks can prevent falls and improve the quality of health-care. A wearable biofeedback system that can monitor the risks of falls, and train users to improve their balance and stability will enhance significantly the quality of life in the elderly and will prevent many injuries caused by falls. It can further be used as a clinical tool for objective assessment and evaluation of gait, and risks of falls.

Wearable health monitoring systems offer tremendous potential to improve our lives, yet a number of hurdles must be cleared before they become a reality. The processors and architectures that enable this technology demand a large amount of energy, requiring sizable batteries. The ultimate objective of this CAREER research is to create programmable architectures which require no battery and can rely on energy harvesting techniques. To achieve this objective, we seek to reduce power consumption of the processing units by more than two orders of magnitude. This power reduction is accomplished through novel, transformative methodologies to compose and configure programmable, ultra low power, granular decision-making architectures. New algorithms, tightly coupled with signal processing for power reduction in communication blocks are investigated. Research methods to activate and deactivate sensors with the granular decision making architecture attempts to further reduce the power consumption. The methodologies are validated and refined using two clinical case studies.

This project has the potential to dramatically improve the quality of health monitoring practice and medical research, empowering numerous applications that are not currently feasible. Growing demand for health care monitoring applications requires students, engineers and health care professionals to design, develop, deploy and operate wearable systems. This project provides a multidisciplinary platform to realize the educational objectives of developing wearable computers, involving high school students and teachers as well as undergraduate and graduate students. Outreach programs are offered through short courses and lectures to local undergraduate societies (SWE, Honors and IEEE), K-12 students, health-care professionals, and local industries.

Researchers propose a self-calibrating integrated approach that operates at the hardware, signal processing and user interface levels to adapt to the new recording session with the least burden on the user as outlined here: 1) The hardware and circuit level approach, where the aim is to find input contact mismatch by injecting a reference signal of known amplitude and observe the common-mode rejection ration (CMMR) of the circuits and electrodes. Artifacts of this reference signal manifest themselves when the electrode coupling is worsening. 2) Employing the signal processing calibration techniques to resolve the strong variation in electroencephalography (EEG) signals from one session to another. Specifically, the research team proposes adaptive training algorithms to utilize relevant information from prior recording sessions to shorten or even omit the calibration time for the next session. 3) Creating customizable user interface in order to produce a more user friendly interface that create less burden on the user. More adaptive user interfaces will lead to more comfortable use, higher transfer rate and better accuracy in realization of the user intents. Researchers plan to develop an inexpensive, easy-to-wear, and low power brain computer interface (BCI) system that uses dry-contact EEG electrodes and can be connected to the computer via Bluetooth and is suitable for real-time applications.


EEG systems have been around for a relatively long time and their applications have been mostly inside the laboratories. However, BCI applications can potentially include any real-world interaction in our daily life. The introduction of low profile, and inexpensive BCI devices with the size of a cellphone and comparable prices create opportunities for new applications controlled with our thoughts, expressions and emotions. For instance, with the rising incidence of chronic diseases, a major health care application for BCI self-calibrating devices is wearable in-home assessment systems to quantify the existence of symptoms or effectiveness of treatments for brain deficiencies through long term EEG recording and analysis. BCI technology has great potentials to become the most common communication alternative for users interacting with computers. For instance, BCI devices are capable of emerging in the gaming industry. It enables the consumers to experience an entirely new form of human-machine interaction by eliminating the conventional joysticks for gaming, entertainment, navigation and rehabilitation.

The rapidly advancing field of wireless technologies and mobile health (mHealth) is likely to play an important role in the future innovations that will help in transforming healthcare to be more individual-centered with a focus on prevention and wellbeing. Yet this nascent field requires advances and innovations ranging from theoretical concepts to large scale implementation.

Intellectual Merit: This project will provide funds to enable students to attend the conference. The participating students will benefit from exposure to computational thinking and advances in in wireless health technologies that utilize real-time data acquisition and inference to support just-in-time interventions. At the same time, they will also learn of the gaps in basic science as well as engineering that need to be addressed in order to achieve the anticipated benefits of the wireless technologies and mHealth. In addition, meeting the leaders in the fields will provide the students with models for selecting and addressing the key difficult problems that need to be solved. Finally, during the workshop, the participants will be meeting their fellow students, giving them an opportunity for scientific networking, so important in a rapidly advancing field.

Broader Impact: Wireless and mobile technologies are likely to have significant societal impacts in a variety of domains. There are indications that these fields will likely contribute significantly to revolutionizing healthcare, from clinical trials to healthcare delivery. These technologies are already changing the social interactions among individuals and groups. Among the most important potential benefits of these technologies are their impacts on bringing healthcare and education to underserved populations. This is reflected in the data suggesting that mobile technology frequently provides the only way for underserved populations to connect to the Internet and interact with the healthcare system. Students working in the field will have the opportunity to accelerate these trends. To achieve these goals the proposer is planning to emphasize the recruitment of women and minority students to attend the symposium.

Ultra-Low Power Inertial MEMS for Pervasive Wearable Computing

Brief description of project Goals:

The project enables ultra-low power accelerometers and gyroscopes for wearable computers with prolonged battery lifetime.

Abstract:

Nontechnical
Advances in technology have led to the development of wearable sensing, computing and communication devices, enabling a large variety of new applications in several domains, including wellness and health care. Monitoring human movements and motor functions perhaps is considered one of the most important applications. Despite their tremendous potential to impact our lives, such systems face a number of hurdles to become a reality. The enabling sensors often demand a large amount of energy, requiring sizable batteries. This creates challenges for further miniaturization. The goal of this research is to enable ultra-low power sensors and DSP's for wearable computers operating with a very small power budget enabling weeks and months of battery lifetime. The proposed research will empower a large set of applications in health care and wellness domains including gait analysis, fall prevention and monitoring physical exercise. This project will ideally reduce the size and weight of wearable computers significantly, enabling many ubiquitous health monitoring applications. It can dramatically improve the quality of health monitoring practice and medical research, empowering more applications that are not currently feasible. This project targets a very important health care application for gait monitoring. Considering the importance of wearable gait monitoring applications and our efforts in reducing the form factor of the sensors that will justify their true ubiquitous use, semiconductor companies will produce billions of chips for wearable computers.

Technical
The proposed research takes advantage of novel electromechanical designs and state of the art micromachining technologies to fabricate contact-based (full or tunneling) inertial sensors with overall dimensions in the hundreds of microns to a few millimeters. Such devices are essentially comprised of a number of acceleration switches requiring a small bias voltage of around 1V and no steady current flow to operate. The output of the sensor is turned ON/OFF by connecting/disconnecting the bias voltage to the device output electrode depending on the accelerations and/or rotation rates the device observes. This is contrary to the existing inertial sensors that provide an analog output requiring significant further processing in the analog domain with a power budget of > 1mW which turns out to be the bottleneck. The proposed new class of devices can be directly interfaced with digital readout/control electronics. The proposed research will also enable a new set of methodologies that co-jointly perform the signal processing and optimize the power of sensors and digital circuitry by controlling the sampling frequency and bit resolution of sensors in real-time. The proposed research will be validated in the context of an important health monitoring application: gait analysis using wrist-worn and shoe-worn sensors.

Electronic-tattoos (e-tattoos) are ultra-thin, ultra-soft sensors and electronics that can noninvasively adhere to human skin like a temporary transfer tattoo. Compared to the state-of-the-art wearable electronics, e-tattoos offer several exceptional characteristics. First, they conform to the skin and create a tight contact with the human body enabling robust signal measurements. Second, they may allow the skin to breathe eliminating the adverse effect of traditional adhesive patches. Lastly, they do not constrain natural skin motion hence present high degrees of comfort for the user. In other words, the user may "put it on and forget about it". E-tattoos are poised to enable new opportunities for the next generation of ubiquitous, unobtrusive and cost-effective health and wellness monitoring impacting the national health, bringing personalized care the individuals need to their homes. A strategic education and outreach effort focuses on broadening the participation of underrepresented groups in science and engineering via a year-long undergraduate research experience with enhanced graduate school preparation in partnership with Texas A & M University EnMed program and University of Texas-Austin NASCENT NSF Engineering Research Center.

Emerging bio-impedance sensing offers new paradigms to capture a number of important physiological signals including heart rate, respiration rate and blood pressure, all around the human wrist. As the most dynamic body part, the wrist is under constant movement. The major challenge in bio-impedance sensing is the negative effects of motion artifacts that corrupt the data, degrade the signal fidelity, and prevent decision making with sufficient confidence. Our project leverages ultra-thin and ultra-soft e-tattoos for bio-impedance sensing on the wrist because e-tattoos enable the most intimate but noninvasive coupling for the electrodes and the human skin, even under severe skin deformation. Our project also explores software reconfiguration methodologies and machine learning techniques to further address the challenges. In particular, we investigate: 1) design and development of an array of submicron-thick, skin-conformable graphene electrode tattoos for the first time, and 2) novel reconfiguration techniques that would eliminate or reduce the noise associated with motion artifacts and enhance the signal fidelity.

This project is to support doctoral student travel and participation for student travel support and participation in the doctoral mentoring program at International Conference on Biomedical and Health Informatics is now on its 5th edition, while the annual International Conference on Wearable and Implantable Body Sensor Networks. These collocated conferences provides a supportive scientific forum for students focusing on computing, health informatics and engineering in the area of human health. The International Conference on Biomedical and Health Informatics and International Conference on Wearable and Implantable Body Sensor Networks will provide a forum for expert and peer critique of students' research with the goal of improving their science. Student participants will also have the opportunity to receive networking support and career advice from internationally-recognized experts. Overall, the travel support brings together students with experts who might not otherwise engage with one another and engage in multidisciplinary groups targeting science the area of human health.

This proposal supports doctoral students by providing a doctoral forum focused on encouraging students to learn from peers and experts from multiple perspectives to understand how innovative computing, informatics, engineering and biomedical science combine to make the most impact in the area of human health. In addition, student participation in a roundtable provided at the conference enables students to explore their science and see different career paths for researchers in this area. The International Conference on Biomedical and Health Informatics and International Conference on Wearable and Implantable Body Sensor Networks conference exposes participants to different scientific disciplinary approaches, supports networking with conference attendees and is designed to support the development of the next generation of scholars in this area and create a bridge between the scientific and health disciplines.

No abstract provided

PROJECT SUMMARY/ABSTRACT The objective of this project is to create an unobtrusive, wrist-worn, cuff-less blood pressure monitor for measurement and identification of nocturnal nondipping hypertension. The investigation includes extensive validation with state-of-the-art ambulatory blood pressure monitors at nighttime in presence of heterogeneous treatment paradigms. Cardiovascular disease (CVD) is one of the major causes of ailments worldwide. Hypertension alone affects one in three adults according to the World Health Organization. Therefore, monitoring blood pressure has become a critical part of healthcare as it is known to be linked to many CVDs. Traditionally, clinical practitioners have relied on the mercury-based (or digital equivalent) inflatable cuff-based sphygmomanometer. However, the nature of the device allows for only infrequent measurements and its somewhat invasive nature and associated discomfort prohibits additional nocturnal measurements. There is certainly a value to measuring blood pressure continuously in the natural context of the user?s environment, in particular during sleep, without being disturbed by the instrument. Our proposed technology can provide a wealth of information to physicians, help identify certain short-term dynamics/variations of blood pressure, and allow effective monitoring of response to medication, among other things. Nocturnal measurements provide additional prognostic value in identifying risk. Despite these benefits, no wearable, non-invasive device for continuous blood pressure monitoring exists on the market simply because none have been reliable enough to be considered clinical grade. This project aims to develop a robust and reliable blood pressure monitor in the form of a wrist-worn device that uses bio-impedance sensors, and for the first time, demonstrate clinical grade reliability. These sensors measure pulse wave velocity (PWV) along with several other derivatives for cardiovascular parameters including heart rate and blood volume changes in arteries, which correlate with the blood pressure. The system will incorporate clever hardware design to localize underlying vasculature and focus on arterial sites for enhanced accuracy. The device will include a motion sensor to take into account the user?s movements and motion artifacts, the contact quality, and reliability of the measurements. Advanced machine learning techniques, leveraging both general and personalized models, will be developed to convert bio-impedance measurements to blood pressure. This project aims to then validate the system and analytics in both a healthy patient cohort and a hypertensive cohort, learning the impact that nocturnal ?nondipping? hypertension and anti-hypertensive treatments have on PWV/other cardiovascular correlates and blood pressure estimates. After decades of relying on the inflatable cuff- based technique, this system could represent a significant change in how we measure blood pressure.

Texas A&M University (TAMU) and University of Texas (UT) proposes a smart and miniaturized patch using graphene-based electronic tattoos (e-tattoos) and a suite of algorithms to extract core body temperature to be used to detect pre-symptoms of infection, with significant utility in understanding and controlling the spread of respiratory and non-respiratory viral infection including coronavirus COVID-19. Skin temperature plays an important role in detecting pre-symptoms of infection. The project provides three intellectual merits: 1) It creates a novel structure that intelligently interfaces a fully flexible graphene-based e-tattoo to rigid printed circuit board using thin film permanent or current-controlled magnets to avoid breakage and for improved mechanical robustness for unobtrusive skin temperature sensing. 2) The project also creates machine learning and deep learning algorithms that leverage the physiological times-series acquired from sensors to predict the core body temperature and will lead to determining pre-symptoms of infection while handling noisy data and enabling personalization of the computational models for each individual using the concept of denoising autoencoders and meta learning. 3) The project creates various techniques to address the real-time operation of the proposed prediction algorithm based on deep learning on low power microcontrollers (MCUs) including methods that strictly use fixed point operations. Given the slow rate of change in the physiological signals and their sparsity, this project will leverage differential sensing over various time scales. These signals can be processed by simplified deep neural network architectures with reduced mathematical operations that facilitates running it on the MCUs for detection.

The broader impact of this project includes a direct response to the COVID-19 pandemic, aiming at protecting healthcare workers and patients through creating novel sensors with significant utility to generate actionable information. Additionally, in light of the growing interest in wearable electronics, this pioneering research effort at the intersection of software, hardware and systems on unconventional e-tattoo platforms can result in breakthrough in data mining and intelligent sensor architecture for mobile health, fitness and computing enabling a larger number of applications. The proposed novel sensing paradigm will provide opportunities for semi-conductor companies to consider new market opportunities and manufacture billions of chips.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.