Ramesh Balasubramaniam, Ph.D. - US grants

The perception of rhythmic patterns of events in time is central to our ability to find meaning in the sounds of language and music: the basis for much of human culture and communication. We do not passively receive temporal patterns, but actively engage with them by using a repeating 'pulse' or 'beat' to form an essential scaffold for our perception of time. This ability might be most obvious when expressed through dance, or simply tapping a foot to music, but it has deeper importance for how we comprehend sound even in the absence of movement. The scaffold provided by the beat cycle enables listeners to predict upcoming events, allowing more efficient encoding and learning of sensory patterns. How does this important perceptual mechanism work? New evidence suggests that perceiving patterns in sound doesn't depend only on the auditory system, but also involves activation of the motor system, even when the listener is not moving. This proposal tests the provocative and potentially transformative idea that motor planning activity is not only to help us move, but is also necessary for perception of patterns in the sounds we hear. This research has many potential societal benefits in both education and medicine. An understanding of the auditory-motor interactions underlying rhythm perception could explain a growing number of findings suggesting an important link between beat perception and language, including the development of reading in children, the perception of speech in noise, and attention, and may help drive improved educational interventions. The results could also provide a brain-based explanation for the growing use of rhythmic music in the treatment of movement disorders such as Parkinson's disease and possibly guide development of enhanced therapies and diagnostic tests.

This proposal addresses a critical, and difficult, open question within auditory cognitive neuroscience: Does motor activity play a causal role in beat perception and if so, what is that role? Establishing this would be a transformative breakthrough in our understanding of the perception of time. While there is strong existing evidence that motor regions are active during beat perception, the dynamic functioning and interaction among parts of the cortical network supporting beat perception is not fully understood. In particular, a causal role of motor activity on auditory processing has not yet been demonstrated directly. This program of research directly examines whether motor planning regions influence processing in auditory cortex and whether a dynamic network is activated during beat perception. To achieve these objectives the investigators use two interlocking approaches: 1) Advanced quantitative methods of cortical source-resolved electroencephalographic (EEG) brain dynamics during beat perception tasks to identify regions in the brain whose activity patterns mirrors the endogenously perceived beat and to examine the directional flow of influence between these beat perception areas and other auditory processing areas; 2) Non-invasive transcranial magnetic stimulation (TMS) to transiently suppress and/or facilitate activity in beat perception areas.

This project will make a new robotic exoskeleton arm with integrated virtual reality and eye-tracking facilities available to researchers for the study of human embodied and situated cognition at UC Merced, the tenth and newest campus of the University of California system. The equipment would enable cutting-edge research in the study of human cognition using tools from robotics benefitting a range of Faculty and students across various departments including Cognitive & Information Sciences, Electrical Engineering & Computer Science, Mechanical Engineering and Digital Humanities. The equipment will catalyze research in several directions including 1) Human Motor Control & Action Dynamics with applications for physical rehabilitation 2) Embodied Cognition and Action 3) Brain-behavior interactions 4) Fundamental human-inspired robotics research 5) Human-machine interactions 6) Language, Communication & Gesture. The robotic arm will provide a sophisticated platform to monitor and manipulate the upper limb, providing a broad range of hand and joint-based kinesthetic information and gaze information. With built in virtual/augmented reality, the robot arm system will allow for altering visual information to be presented to human participants while manipulating their arm movements. The work will help build institutional capacity in an area of strategic focus at UC Merced and will result in direct collaborations between scientists and students across various departments on campus. The impact of the training resulting from the equipment is expected to be particularly broad at UC Merced due to its exceptionally diverse student body and the large number of programs focused on increasing the participation of underrepresented minorities.

Researchers including the PI: Balasubramaniam and Co-PIs Spivey and Carpin also plan on perturbing neural activity using techniques such as Transcranial Magnetic Stimulation (TMS) and recording brain activity during the perturbation of arm movements using the robotic device. The transformative aspect of this research comes from its ability to observe and manipulate brain activity and complex motor behavior simultaneously.

The world is bursting with data, not just in sheer amounts of it, but also in terms of complexity. Interdependencies among variables abound, and their relationships can change over time in intricate, nonlinear ways. Such complexities are common in nature and intelligent systems have evolved in biological organisms to adapt to these interdependencies and nonlinearities. More recently, engineers have begun to build intelligent systems for applications in health, security, and industry that can similarly adapt. The scientists who study intelligent adaptive systems in nature, as well as the engineers who build them in the lab, are increasingly in need of conceptual and technical abilities to deal with large, complex systems and datasets. These abilities provide a common basis for exchanging hypotheses and theories among mathematicians, physicists, biologists, cognitive scientists, computer scientists and engineers; all of whom work on common problems of adaptation, learning, regulation, and prediction. This National Science Foundation Research Traineeship (NRT) award to the University of California, Merced, will help the next generation of PhD students make interdisciplinary breakthroughs in theories and applications of intelligent adaptive systems. The project anticipates training 100 PhD students, including 50 funded trainees, from doctoral programs in applied mathematics, cognitive and information sciences, electrical engineering and computer science, mechanical engineering, physics, and quantitative and systems biology.

Prior research in cybernetics, connectionism, and complex adaptive systems focused on general principles of intelligent adaptive systems that cut across disciplines and domains. The NRT program will advance the next wave of research in this area, by delving more deeply into principles of learning and adaptation as they manifest across a wider range of biological, human, and technological systems. The training program includes an intensive computational basecamp, custom course modules on intelligent adaptive systems, lab rotations, communication skills development workshops, and industry networking opportunities. Taken together, these NRT activities will enable the trainees to achieve conceptual and technical capabilities for dealing with large, complex datasets. All NRT trainees will have the opportunity to learn about entrepreneurship, network with industry mentors, engage in professional development, and engage with the local community to educate, disseminate research, and develop outreach partnerships. The NRT program will transform the capacity for interdisciplinary research and education at UC Merced. At the institutional level, the NRT program will serve as a model for collaborative, interdisciplinary graduate education. An extensive recruitment plan will connect with and enhance resources and programs at other UC campuses and a number of Hispanic-Serving Institutions to increase the diversity of scientists and engineers working on intelligent adaptive systems. Finally, the NRT program will have a direct and transformative economic impact in California's Central Valley, by fostering a culture of innovation and higher education in under-privileged communities.

The NSF Research Traineeship (NRT) Program is designed to encourage the development and implementation of bold, new potentially transformative models for STEM graduate education training. The Traineeship Track is dedicated to effective training of STEM graduate students in high priority interdisciplinary research areas, through the comprehensive traineeship model that is innovative, evidence-based, and aligned with changing workforce and research needs.

This award will support an interdisciplinary workshop to study how humans and intelligent machines, such as robots, may behave while physically interacting with each other, in contrast to situations where they only exchange information. These questions will become increasingly important as humans begin to come into frequent physical contact with intelligent machines, such as when navigating shared spaces, riding in or walking near autonomous vehicles, or interacting with caregiving or physical therapy robots.

This workshop will gather experts from cognitive neuroscience, behavioral science, dynamics and control, robotics, optimization, and computer science to define key challenges in analyzing the dynamic behavior of systems of embodied human and machine intelligences. Due to rapidly developing capabilities in machine learning, and wide scale deployment of autonomous and intelligent robotic systems, humans in the near future will regularly find themselves physically interacting with intelligent technology. Such intelligent technology might include home assistance robots, physical therapy robots, autonomous vehicles, and smart buildings. The dynamics of physically coupled of humans and intelligent machines have so far received little study, despite the potential for unexpected behavior.