Joachim Vandekerckhove - US grants

This project will combine two types of data analysis strategies that are common in different fields. In cognitive psychology, the state of the art is cognitive process modeling. Data are analyzed by fitting mathematical representations of cognitive functions to data and interpreting the obtained parameter estimates. In psychometrics, the most common form of data analysis involves latent variable modeling. Batteries of small tests, each individual test imperfect, are jointly analyzed to uncover unobservable underlying factors, such as general intelligence or specific abilities. Cognitive modeling succeeds in extracting more information from data, whereas psychometrical methods are useful for pooling information across tasks or participants. Combining these two traditions involves the formal challenges of applying latent variable structure to cognitive model parameters, integrating the mathematical assumptions of both strategies, and investigating the effects of those combined assumptions. The project also involves technical challenges, such as implementing the methods in software. A new hybrid method called cognitive structural equation modeling will be applied in a retrospective analyses of data on cognitive executive functions and data on facets of working memory. Additionally, a cognitive structural equation model will be used to investigate the stability of participants' behavior in cognitive tasks over time.

The new method will be particularly well suited for the simultaneous analysis of different cognitive tasks in order to uncover underlying structure in participants' aptitude in the tasks. Improvements in psychological measurement are potentially useful in a variety of contexts, ranging from fundamental research in perception, cognition, memory, decision making, emotion, and development, to applied measurement in educational testing, job selection, and psychodiagnosis. Software developed as part of this project will be made freely available to researchers.

The differential rate of publishing between positive and negative results, which has been called publication bias, is of increasing concern in the social and behavioral sciences. This research project will develop a new approach to meta-analysis that explicitly takes into account the possibility of a biased publication process. Meta-analysis has been instrumental in interpreting the claims made in the academic literature. However, academic journals, especially in the social and behavioral sciences, seem to strongly prefer manuscripts that posit the existence of an effect rather than non-significant outcomes. This hinders classical meta-analysis methods because the aggregate of a biased set of empirical results will be biased as well. The new approach will allow for better aggregation of published results and will provide a more accurate view of the effect of various experimental manipulations and treatments. Software will be developed and published that implements this approach for a variety of situations.

This research project will develop a new approach to meta-analysis called "statistical mitigation" that combines behavioral models with state-of-the-art statistical methods. The approach will be based on a Bayesian model averaging technique in which effect size estimates are computed using a set of plausible selection models and averaging across these selection models. With this approach, it will be possible to isolate the signal of true effects within the noise of measurement error. The investigator will test the method under various circumstances, compare the new approach to existing methods for inference in the presence of publication bias, and perform simulations to assess the efficiency of the method. With a single approach to meta-analysis, researchers will be able to account for the possibility of publication bias, confirm or disconfirm null and non-null hypotheses, and do effect size estimation.

This award provides support for student participation at the 2015 Annual Meeting of the Society for Mathematical Psychology, to be to be held in Newport Beach, CA in July 2015. The annual meeting will advance discovery and understanding of theoretical problems in psychology and the social sciences. It will create an environment in which mathematically oriented social scientists can grow and flourish. It will promote training and learning of new models and methods for analyzing behavioral data and make possible the broad dissemination of new findings important in all areas of psychology, as well as economics, political science, sociology, and anthropology. Support for student travel will allow the organizers to involve more individuals from underrepresented groups.

Mathematical psychology is interdisciplinary by nature, bringing together social scientists, statisticians, mathematicians, and computer scientists to work on problems critical to behavioral and cognitive scientists. The 2015 annual meeting will include three symposia on new developments that have not been featured at recent meetings: (1) Dynamics of adaptive rationality; (2) Systems factorial technology; and (3) New developments in the analysis and modeling of naturally occurring data sets. The annual meeting also will include a half-day professional development symposium organized by the Women of Mathematical Psychology and a post-conference networking event with potential industrial partners.

Understanding the role of changes in brain activity over time is extremely difficult. Moreover, relating these changes to the psychology of decision making is even more challenging, but important to understand to predict and explain behavior. In particular, brain activity can vary in relationship to the speed of a behavioral response, but these variations are difficult to measure systematically. The goal of this project is to measure brain electrical activity together with behavioral data in order to develop new methods to statistically model the relationships and to aid in detection of subtle brain changes occurring with behavior. The project will result in a new method for analyzing these complex relationships and allow for better combination of different forms of data generally. This will provide a more accurate view of the effect of experimental manipulations and treatments. All analytic procedures will be extensively documented along with the experimental data and these will be freely available online.

A crucial tool in this project is the new technique of joint modeling of behavioral and physiological data. An advantage of joint modeling that has thus far been underexploited is the capacity to construct genuine neurocognitive models that are informed by both behavioral and neural data. Indeed, joint estimation opens up the possibility to construct new models whose parameters are only estimable given more than one type of information. This project will lead to the development of a multimodal sequential accumulation model that makes predictions about the combination of reaction time, accuracy, and EEG data, and that allows for conclusions not possible from either type of data individually. Specifically, the project involves experimental studies to generate data that will, in combination with the new statistical framework, allow the disentanglement of parameters of a cognitive model that cannot be estimated without the use of both neural and behavioral data. The parametric neurocognitive model will involve specific neural markers that connect behavioral parameters to EEG activity measures. The newly collected data will also provide a direct test of the classical modeling assumption that visual encoding, decision-making, and executing a motor response are sequential processes.

This National Science Foundation Innovations of Graduate Education (IGE) award to the University of California-Irvine will test a cascading mentorship program for science, technology, engineering, and mathematics (STEM) doctoral students. A centuries-old model of doctoral mentoring assumed that students learn by working alongside their faculty advisors at the lab bench. However, research as evolved into a team activity with advisors and students serving different roles and rarely working together at the same task. In this environment, students typically learn most research skills from advanced graduate students and postdoctoral researchers rather than their faculty advisors. Cascading mentorship describes such a model of mentoring from a rich network of faculty and peers. Doctoral students who receive cascading mentorship are much more likely to develop strong research skills than those who do not regardless of the quality of mentoring received from their primary faculty advisors. However, not all students are trained in large, well-functioning labs with ready-made cascading mentorship networks. In addition, factors such as shared or dissimilar cultural identity with the majority of faculty and differences in first-generation students’ access to faculty mentors, friends, or family members with doctoral degree experience may result in differences in the quality of mentoring. Thus, when doctoral programs fail to provide rich networks of cascading mentorship, it is the students from minority communities who are likely to suffer most.

In this project, approximately 65 advanced graduate students and postdoctoral researchers in cognitive science will be trained to lead cascading mentorship groups of five to ten doctoral students each. The groups will use proven pedagogical methods to help members plan their research, organize their time, and learn to produce the forms of writing on which academic success depends such as: literature reviews, funding proposals, research reports, etc. This project will address research questions related to four scale-up challenges: 1) group leader effects, i.e., whether the approach works when these mentorship groups are led by advanced graduate students rather than by a faculty member; 2) implementation cost, i.e., whether these groups can be supported in a way that is affordable for most doctoral programs; 3) long-term effects of cascading mentorship over a period of years in a variety of domains including: writing quantity and quality; progress through the graduate program; mental health and well-being; and 4) broad suitability, i.e., whether the model is equally effective for all students or whether only a subset of students benefit. Results from this project will help departments decide whether to adopt a cascading mentorship model program-wide or offer this model as an option for a subset of students.

The Innovations in Graduate Education (IGE) program is focused on research in graduate education. The goals of IGE are to pilot, test and validate innovative approaches to graduate education and to generate the knowledge required to move these approaches into the broader community.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.