Mitra J. Hartmann - US grants

A most basic form of human social interaction is the physical cooperation necessary to perform a manual task in pairs (dyads) or groups. This project will explore how the human sensorimotor control system implements cooperative motor control. In preliminary work, the project team has demonstrated that when a dyad embarks on a repeated simple manual task requiring speed and accuracy, motor strategies quickly arise that not only differ significantly from individual strategies on the same task, but also provide better performance. This result runs counter to conventional wisdom that high accuracy tasks are best performed by one individual alone. Nevertheless, from both evolutionary and ontological perspectives, the result is reasonable: humans are social animals and have developed sophisticated ways of working together physically. Motor interactions represent a social communication mechanism distinct from facial expression, gesture, and spoken language.

The project team encompasses cognitive science, neurobiology, robotics, and sensorimotor control, and will:

Investigate the language of physical communication between two or more individuals as they develop a cooperative strategy to perform a mechanical task. Identify channels of this communication, for instance modulation of arm stiffness. Investigate the adaptation that underlies the emergence of cooperative behaviors during physical communication, as the participants negotiate, compromise, specialize, teach and learn, or in some other way arrive at an effective cooperative behavior.

Investigate cognitive influences on cooperative behaviors. Determine the extent to which the cooperative behaviors reflect cognitive influences on motor control, as opposed to implicit or inherent biomechanical properties of the sensorimotor system.

Investigate the emergent behaviors as specifically social phenomena. Assess their extension to groups sizes of more than two, and the social aspects of adaptation, such as the effect of errors or "breaches of trust." Investigate the substitution of an automated partner (robot) for a human partner.

Investigate the factors at the sensorimotor level that allow dyadic motion control to optimize performance better than individual motor control. Hypotheses include reduction in delays associated with the triphasic burst pattern of muscle activation, and the partitioning of motor noise into separate spaces.

Broader impacts include: improving our fundamental understanding of therapist/patient interactions during physical or occupational therapy, many aspects of which are repetitive dyadic physical interaction. The work may also lead to better ways to make use of the social dynamics between individuals in physical interaction, which would be relevant to situations such as hands-on teaching/learning a in sports training or helicopter flight training, shared control of teleoperators or of unmanned aerial or underwater vehicles, and shared control of minimally invasive surgery or telesurgery.

Neuromechanical Models of the Rat Vibrissal System Mitra
J. Hartmann, Michael A. Peshkin, Northwestern University

Animals use movements to acquire and refine incoming sensory data to construct meaningful representations of the environment. This process is often called "active sensing." During exploratory behaviors, each movement an animal makes aids in the extraction of task-relevant sensory data. As yet, however, neuroscientists have little understanding of how the body and brain work together to acquire, encode, and process the sensory data generated through movement. To study the neuromechanical principles that underlie active sensing behaviors, the investigators will construct an active sensing system in hardware based on careful modeling of a well-understood sensorimotor system: the rat vibrissal (whisker) array. The rat whisker system is an ideal model for studying active sensing behaviors. When exploring their environment, rats sweep their whiskers back and forth in the air and against objects at frequencies typically between 5 and 12 Hz. Using this whisking behavior, the rat can extract accurate information about an objects spatial properties, including size, shape, orientation, and texture. The core of the project involves (1) characterizing the mechanics of rat whiskers and natural whisking movements, both when moving freely in air and when in contact with objects (2) constructing an array of actuated, biomimetic (robotic) whiskers with sensors at the base (3) developing models to interpret the spatiotemporal patterns of whisker sensory activation (both real and robotic) to extract object features. The results will directly generate hypotheses about how information is represented in the rat nervous system, and shed light on the many hundreds of neural recordings from rat somatosensory cortex (barrelcortex) that are performed each year. This project begins to establish rigorous mathematical models for sensory encoding in the whisker system that may generalize to other sensorimotor pathways. In computer science, the research may inspire studies on unsupervised 3D object recognition using non-optical sensors. The project contributes to the interdisciplinary scientific training of both graduate and undergraduate students, brings a quantitative engineering approach to neuroscience, and directly complements coursework in Neural Engineering being developed at Northwestern University.

Rats have approximately sixty large facial whiskers that serve as exquisitely sensitive tactile sensors. Tactile information from the whiskers is carried to a brain structure called the trigeminal ganglion, and then to structures called the trigeminal nuclei. This pathway is roughly analogous to the pathway that carries information from the human skin to the brain. Professors Hartmann, Memik, and Ogrenci -Memik are using the rat whisker system as a model to study the process by which animals acquire and refine incoming sensory information to construct representations of the world around them. The research iterates between neurophysiological recordings in the trigeminal nuclei and computational modeling of the observed neural responses. Because the nervous system is massively parallel, the models quickly exceed the limits imposed by conventional computers. The models are therefore being implemented with Field Programmable Gate Arrays (FPGAs), which can effectively replicate the immense parallelism of brainstem circuitry. Notably, the models are built to process data from both real and robotic whiskers, and can be used to generate predictions about physiological responses in the real animal.

This work has the potential to have an impact in industry as well as academia. From an industrial perspective, it is likely to inspire studies on unsupervised 3D object recognition using non-optical sensors. Such sensing ability may be useful in a variety of specialized environments. For example, search and rescue robots operating with limited or no vision could potentially use hardware and algorithms similar to those developed here. In neurophysiology, the work is shedding light on processing algorithms common across modalities (vision, audition, somatosensation and proprioception). The work also contributes to strongly-interdisciplinary training at the post-doc, graduate and undergraduate levels, as well as to minority student recruiting. The work also complements graduate-level courses developed by the PI and Co-PIs.

When you explore an object using your hands, or you change your footsteps based on the type of surface you are walking on, your sense of touch is seamlessly combined with the movements you make. How does your brain accomplish this feat? This is a large, open question that is difficult to answer using human studies alone. This project will investigate this question by studying the rat whisker (vibrissal) system. Rats move their whiskers back and forth to touch and explore different objects, much as humans use their hands. The tactile information is conveyed from the base of the whiskers through a series of processing stages in the brain. This research project focuses on understanding the processing that occurs in the second stage, in a brain structure called "spinal trigeminal nucleus interpolaris" (SpVi). In the first part of the project, computer simulations and robotic models will be used to test the plausibility of the different types of processing that might occur in SpVi. In the second part of the project, technology to measure the precise contact times and locations of a rat's whiskers with different objects will be developed. Finally, in the third part of the project, electrical signals from neurons in SpVi, will be recorded and correlated with the animal's ongoing behavior as it touches different objects. The results that emerge will allow the testing of three different hypotheses for the processing that occurs in SpVi. Regardless of which hypothesis(es) is/are found to be correct, the results will improve the understanding of how the brain combines sensing and movement to allow animals to perceive the world. This project will contribute to the interdisciplinary training of two graduate students and to ongoing graduate-level course development at Northwestern University. It will also provide meaningful research opportunities in engineering for at least six undergraduates, with a specific effort made to include women and underrepresented minorities.

The rat whisker system is one of the most commonly-used systems in neuroscience to study how the brain combines sensory information with movement information. To date, however, the small size and rapid movements of the whiskers have made it difficult to quantify their dynamics. This work will quantify how the whiskers move and interact with objects during rat exploratory behavior. Improved understanding of the rat whisker system will provide considerable insight into the general functional principles that govern the neural circuits mediating sensing and control.

A dynamic model of the isolated whisker moving in free air and during object collisions will be developed. This will be incorporated into an anatomically accurate 3-dimensional simulation environment. Finally, behavioral experiments will identify the patterns of mechanical input across the array associated with the rat's ability to distinguish between flat and curved surfaces. The ultimate goal is to determine how the brain might represent the shape of an object based entirely on the patterns of forces and moments at the base of each whisker across the array. The simulation environment will be distributed widely, so that researchers working on any brain structure involved in rat whisking behavior can predict whisker-object contact patterns.

The simulation environment will also be used in conjunction with previously developed robotic systems in a classroom environment and to aid in the recruitment of underrepresented minority engineering applicants. Students will be engaged in understanding how basic mechanical principles must be applied to sensory neuroscience. Simultaneously, the PI will develop a coordinated program for undergraduate research across the McCormick school of Engineering at Northwestern University. The success of these initiatives will be carefully evaluated with the help of Northwestern's Searle Center for Teaching Excellence.

EFRI-BSBA: Bio-Inspired Arrays of Haircell Sensors for Artificial Glabrous and Hairy Skin

PI Name: Chang Liu

Institution: Northwestern University

Proposal Number: 0938007

Abstract

Biological sensors exhibit exquisitely high sensitivity, large dynamic range, and robust and efficient signal reconstruction. They accomplish these even with imperfect and noisy data. Further, most biological sensors are employed as flexible sensor skins consisting of two dimensional, high density, multimodal sensor array. The overarching engineering objective of the proposed work is to develop a flexible, multimodal sensing skin for the sense of touch that exploits biologically inspired principles to achieve high sensitivity, wide dynamic range, and an advanced, highly-efficient signal processing capability. Specifically, we will focus on developing artificial glabrous and hairy touch-sensitive skin ? parallel arrays of multimodal tactile sensors ? along with companion signal processing algorithms. A glabrous skin is largely smooth and consists of normal and shear contact sensors, whereas a hairy skin further consists of exposed hairs for proximal touch sensing. We plan to test-bed the sensors and algorithms in one application: smart, sensorized catheter tips for cardiac surgery procedures (e.g., tissue ablation, internal space mapping, ECG recording), with intended benefits of increased accuracy, reliability, and speed. The proposal seeks transformative interdisciplinary research in the following major areas: (1) The proposal team will develop a silicon-polymer hybrid, flexible sensing skin with multimodal sensor integration by leveraging existing expertise of artificial haircell sensors and multimodal tactile sensors. (2) The team will develop a radical multimodal tactile sensor skin, mimicking glabrous and/or hairy skin, consisting of artificial hairs (whiskers) and other tactile sensing modalities (normal and shear contact, temperature, etc). (3) The team will seek biological inspirations at many levels to increase the sensitivity, widen the dynamic range, and enhance speed and robustness of downstream signal processing. (4) The team will creatively explore the use of active sensing ? sensors that sense the properties of surroundings through movement and active engagement, rather than passive interfaces. (5) The team will perform cutting edge research on signal processing, to face the challenge of massive sensor data and multimodal sensor informatics fusion. Here, bioinspiration will be sought by investigating a model system, early signal-processing stages of the rodent whisker (vibrissal) touch sensor.

The rodent vibrissal-trigeminal system is one of the most important models in neuroscience for the study of sensorimotor integration. To date, however, research has focused exclusively on direct tactile sensation. Recent results from the Hartmann and Gopal laboratories have demonstrated that rat vibrissae have a robust and repeatable mechanical response to airflow. In addition, neurons in the vibrissal-trigeminal system are known to respond to air puffs. These results suggest that the rat may use its vibrissae to detect air currents and determine wind direction. The Northwestern-Elmhurst team will perform mechanical, behavioral, and computational studies to characterize the role of vibrissae in wind-following behaviors, and the vibrissal-related neural response to air currents. These will constitute some of the first investigations of the underlying mechanisms that permit terrestrial mammals to sense and follow the wind. The team will specifically identify the morphological features of vibrissae and their orientation on the mystacial pad that enable flow sensing behaviors. They will investigate the broad hypothesis that differential mechanical deformations of the vibrissae across the mystacial pad can encode a variety of flow parameters. Finally, behavioral experiments will be performed to determine the extent to which the rat uses its vibrissae to sense airflow, and to quantify the movement strategies used during anemotaxis in the behaving animal. The partnership between Northwestern and Elmhurst will provide significant research opportunities for undergraduates; in addition, videos will be developed to teach the fundamental principles of fluid dynamics and biological sensing that underlie this research. The proposed work has potentially large implications for olfactory localization and the structure of the olfactory system, and is likely to lead to the development of novel flow-sensing technologies.

? DESCRIPTION (provided by applicant): We see because retinal ganglion cells respond to light. We hear because spiral ganglion cells respond to sound. We feel because primary somatosensory neurons respond to touch. But what is touch? Whereas light and sound can be characterized by physical parameters (amplitude, frequency, phase, and polarization), the mechanics of touch, and the manner in which primary sensory neurons encode the parameters of touch, are largely unquantified. This is a glaring gap within the entire field of somatosensation, and it occurs because mechanics are difficult to quantify. To close this gap we will use the rat vibrissal (whisker) system as a model to directly relate the responses of primary sensory neurons to the quantified mechanics of touch. Paralleling the increased use of rodents in genetic and optogenetic research, the rodent vibrissal array has become an increasingly important model for the study of touch and sensorimotor integration. In the past few years, our laboratory has made rapid progress in characterizing vibrissal mechanics, and we are now uniquely positioned to determine how 3D whisker deflections and vibrations are represented in the firing patterns of primary sensory neurons of the trigeminal ganglion (Vg) during natural whisking behavior. The central goal of our investigation is to predict the responses of Vg neurons during both contact and non-contact whisking by appropriately combining 3D dynamic and quasistatic models of mechanical signals. Our three aims move from the outside of the rat inwards, from whisker, to follicle, to Vg neurons. In Aim 1, we will develop models of mechanical coding by the whisker, quantifying the 3D mechanical signals at the vibrissal base during both contact and non-contact whisking. In Aim 2, these models will be used to predict responses of mechanoreceptors within the follicle and thus to identify classes of Vg neurons based on the mechanical transformation they perform. Finally, in Aim 3 we will quantify the responses of Vg neurons during natural whisking behavior in awake animals. Exploiting the cell classes identified in Aim 2, and consistent with the modeling of Aim1, we will test the hypothesis that Vg responses are more linearly correlated with mechanical signals during whisking than they are with the geometry and kinematics of whisking behavior. The proposed work will be the first to record from Vg neurons in awake behaving animals while fully characterizing the mechanical input during both contact and non-contact whisking. We aim to solve a large portion of the coding problem for the vibrissal-trigeminal system. Solving this problem will provide a better understanding of what a Vg spike means for more central stages of the trigeminal system, including sensory thalamus and barrel cortex.

? DESCRIPTION (provided by applicant): The long-term objectives of this multi-PI R01 proposal are two-fold. First, the functions of specific parallel neural pathways (lemniscal, paralemniscal) linking brainstem and forebrain trigeminal (V) representations will be revealed. This will be done in the rodent whisker-to-barrel cortex neuraxis, which has become the model of choice for discovery of information processing mechanisms, due to the prominence of barrels in the cerebral cortex of transgenic mice. Yet, there are major gaps in our knowledge of subcortical components that hamper our grasp of the whisker-barrel circuit. These gaps will be filled by applying a set of multidisciplinary tools to studies of the neural control of whisker-related sensation and movement. Our overarching hypothesis is that neural activity in the spinal V subnucleus interpolaris (SpVi, paralemniscal) is necessary for whisker-mediated object detection and orientation responses, while whisker-mediated object identification and discrimination require neural activity in a topographically patterned (barrelettes) V nucleus principalis (PrV, lemniscal). Three Specific Aims employ: a) transgenic mice that lack barrelettes in PrV, but not in SpVi, b) reversible allatostatin- induced silencing of adenovirus transduced multi-whisker responsive SpVi cells or single-whisker PrV cells, and c) anatomical and electrophysiological assessments of the integrity and neurotransmission properties of V brainstem neurons in animals studied in the above 2 Aims. Thus, gene deletion and neural silencing approaches are coupled with parallel validation of the extent to which these 2 approaches produce functional lesions in V brainstem neurons. This permits discovery of components of the barrel neuraxis that are unequivocally responsible for whisker-mediated detection, orientation, identification and discrimination behaviors. Second, this research will als provide technical 'proof of principle' for the potential use of above- listed allatostatin-induced silencing of adenovirus infected neurons to treat human neurological disorders, such as epilepsy, chronic pain, obesity and addiction, where hyperexcitability characterizes defined neuronal populations, reduction or elimination of which could constitute a new treatment strategy. Self-administration of allatostatin could be a transformative treatment option, the efficacy of which will be evaluated here in a simple model system, with an eye towards possible side effects. A collaborative venture is offered with 3 PIs that are indispensable to the accomplishment of all 3 Specific Aims. Dr. Zeigler developed the head-fixed technology required to deliver stimuli to single whiskers, to monitor their movements and to bring such movements under voluntary control. These tools will be used in Aims 1 and 2. Dr. Hartmann developed the head-free technology required to measure and control movements of single whiskers, as well as means to analyze video materials of such. These tools will be used in Aims 1 and 2. Dr. Jacquin's career has been largely devoted to the trigeminal system and brings expertise on transgenic mice, allatostatin/adenoviruses and single unit recording to bear upon the behavioral issues and technology offered here by Drs. Hartmann and Zeigler.

In humans, the sense of touch is closely linked with hand movements. An open question in neuroscience is how the brain combines touch signals (sensory) with hand movements (motor) to create a unified tactile perception of an object. Because of difficulties in studying how the human brain combines these cues, researchers study rats, which use ~60 whiskers to tactually explore the environment. The whisker system is an excellent model to investigate how neurons represent and unify touch and movement, but neuroscientists currently struggle to stimulate the whiskers in a way that imitates the signals obtained during natural exploratory behavior. In this proposal, the investigators will characterize naturalistic patterns of tactile input that the rat's brain evolved to process, and will develop new mathematical tools to describe the environmental features that the rat experiences. The proposed work is scientifically important for two reasons. First, because rodents are the most commonly used animals in neuroscience, these experiments will aid researchers studying many parts of the brain involved with touch and movement. Second, these new mathematical tools will help quantify the sense of touch across species, including humans. The proposed work will have significant broader impacts on science and mathematics education and public outreach. Undergraduate students will contribute to the work, and the investigators will lead Northwestern's Robotics Club to explore engineering applications of whisker-based tactile sensing. The investigators will also continue outreach efforts through the Society of Hispanic Professional Engineers, the Society of Women Engineers, and Chicago's Museum of Science and Industry.

In the fields of vision and audition, the receptive fields of central neurons are tuned to the statistics of the "natural scenes,- to those properties of the stimuli that the animal is likely to encounter in its natural environment. In the field of somatosensation it is challenging to quantify the natural tactile scene, in part because somatosensory signals are tightly linked to the animal's movements. The proposed work aims to begin to quantify the natural tactile scene for the rat vibrissal system by combining careful behavioral monitoring and simulations of rat head and whisker movements. The project has two major goals. The first is to characterize the statistics of the environments that the rat naturally inhabits. In Bayesian terms, this statistical distribution is called the "prior," because it describes the environment's geometrical features, unbiased by the tactile sampling choices of the animal. The second is to quantify the statistics of the environment sampled by the rat, given its choices of head motions and whisk cycle. In Bayesian terms, this statistical distribution is called the "posterior," because it incorporates the bias of the rat when preferentially sampling the tactile scene. This work is one of the first attempts to quantify the statistics of active touch, and it aims to make specific predictions for the receptive field properties that enable spatiotemporal integration. Equally important, this work will develop an appropriate mathematical framework for characterizing the geometry of natural scenes, an essential step towards describing active somatosensation using information theoretic measures.

This project will construct robots in order to understand how animals gather information through the sense of touch and how animals use touch information to perform complex behaviors. The results will be important to both neuroscience and engineering. On the neuroscience side, the results will address how the brain combines information about movement and touch, thereby improving our understanding of stroke and brain injury. On the engineering side, the work will develop novel robots and sensors that use touch to sense object location, shape, and texture, to track fluid wakes in water, and to sense the direction of airflow. These capabilities will improve the ability of robots to work in challenging environments; for example, robots could explore dark areas more easily or provide surgeons with a better sense of touch during surgery. To train the next generation of scientists and engineers, both undergraduate and graduate students will help construct the robots and will explore industry- and government-related applications of whisker-based touch sensing. The research team will investigate technology transfer opportunities in robotics and medicine, flow sensing, instrument placement, corrosion detection, three-dimensional tactile profilometry, and compliance sensing.

The fundamental scientific rationale for the work is that understanding how animal nervous systems process complex sensory and motor information necessarily requires quantification of the input. However, it is currently impossible for neuroscientists to record from all primary sensory neurons involved in a particular sensorimotor behavior. The three stages of this project exploit the whisker system of mammals in an endeavor to completely quantify whisker-based input and early neural processing in the rat (Rattus norvegicus) and the harbor seal (Phoca vitulina). The first stage of work will focus on the development of modular, reconfigurable, artificial whiskers that can sense both touch and fluid flow. The materials, manufacturing, and sensor designs necessary for whiskers at multiple length scales will be investigated and signals from the whiskers will be represented based on known coding properties of primary whisker-sensitive neurons in the trigeminal ganglion (TG). The second stage of work will involve the construction of whisker arrays that anatomically match those of the rat and the seal. These arrays will be used to develop combined hardware and software models of the responses of the entire population of TG neurons. Finally, in the third stage of work, the whisker arrays will be mounted on robotic platforms, and the robots will be put through the same head movements as real animals during natural behavior. This process will allow us to simulate the entire TG neuron population response during complex, natural behaviors. Overall, the project will help unlock the basis by which low-level but powerful neural circuits confer animals with flexibility and resourcefulness in sensing and movement. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a mulitdisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).

Project Summary: The rodent vibrissal (whisker) system is one of the most widely-used models in neuroscience to study how information about movement and touch are combined. During many exploratory behaviors, rats and mice sweep their whiskers back and forth in a rapid, rhythmic motion called ?whisking? to actively gather touch information. Although whisking is rhythmic, rodents can also change how their whiskers move depending on the desired sensory information, and on their particular behavior. Researchers are nearly able to begin to ?close-the-loop? between movement and touch for the whisker system, except for one critical gap: we do not yet have a three dimensional (3D) model of rodent facial musculature. Without such a model, we cannot identify how the rat changes its muscle activity to change whisker motion and acquire particular types of sensory information. We cannot know which whisker motions are fixed via the biomechanics, versus which motions the rat can actively control. We cannot fully understand the motor commands sent to the whisker muscles. The central goal of this proposal is to develop three-dimensional (3D) models of rodent facial musculature that close this gap. We will first use a novel combination of tactile profilometry, histology, MRI, and CT-scans to quantify the anatomy of rodent facial muscles and the follicles that hold the whiskers. Using this anatomy, we will then construct 3D biomechanical models of the whisker muscles and follicles to simulate the motion of all whiskers. These models will be validated and tested in several different complementary software systems, and then be used to test eleven specific predictions for the particular function of each whisker-related muscle. Finally, we will integrate the 3D models of rodent facial muscles with existing models that describe the sensory, tactile side of whisker motion. These combined muscle-sensory simulations will be directly compared with active animal behavior. This work takes a step towards closing the loop between motor action and the sensory data acquired, and helps disentangle the relative roles of biomechanics and neural control during different types of whisking. The proposed work will inform all levels of study of whisker neural pathways, from primary sensory neurons to sensory and motor cortical areas, to brainstem regions involved in controlling whisker motions. More generally, whisking represents a unique window into how volitional control can modulate or override centrally-patterned movement. The transition between varieties of rhythmic and non- rhythmic movement has important implications for the coordination of sniffing, breathing, olfaction, chewing, swallowing, and suckling, and the proposed work could thus shed light on the neuromechanical basis for some pediatric and geriatric dysphagias.