William Howard Batchelder - US grants

Researchers in the cognitive and brain sciences are generally concerned with mental functioning and the workings of the mind. Unfortunately, the cognitive processes that these researchers wish to study, such as attention, learning, and memory retrieval, are not directly observable. Thus, unlike physicists and chemists who have specific instruments to measure observable phenomena, cognitive scientists have no direct scales by which to measure the functions of the brain. This project will develop new measurement tools for the study of mental processes. In particular, the project will develop a scientific method of measurement called multinomial modeling. Multinomial models are relatively simple statistical models that can transform human performance data into measures of cognitive factors. These models have been successful in overcoming measurement problems in other areas of science, such as statistical genetics. In general, this project will create and test a number of multinomial models in crucial areas of human cognition. Scientists will then be able to use these models to explore more effectively many theoretical and practical issues in the areas of cognitive science and neuropsychology. For example, multinomial models may enable us to determine if brain dysfunction, such as in schizophrenia, occurs at the encoding stage of information processing or at some higher level. It may be possible to discover if the memory deficits associated with old age and disease, such as Alzheimer's disease, affect people's ability to store information or just their ability to retrieve it. In addition, these models will be able to pinpoint the precise effects of certain drugs and other disruptive variables on different aspects of cognitive functioning.

Social measurements derived from informant judgements are widespread and improvements in their quality are critical to advancing social science research. Examples of these measurements, drawn from various areas of social science, include: ratings of occupational prestige in sociology, rankings of friendship in social networks and social psychology, determinants of group preferences in social choice, judgments of the similarity among kinship terms in anthropology, and judgments of the seriousness of crimes in political science, etc. The long range goal the this research is to contribute scientific methods to improve the quality of social measurements that depend upon aggregating the judgments of informants. Social concepts such as cultural beliefs and social norms are defined by social conventions, with no outside criteria for comparison. Measurements must be inferred from the pattern of responses among informants, obtained in field settings with limitations on time, facilities, and number of informants. Combining methods from foundations of measurement and statistical modelling, this research probes the fundamental logic underlying two problems basic to social measurement: (1) How can one meaningfully infer item attributes from responses of informants, each of whom may be calibrated on a different scale ? Meaningful social attributes should satisfy some sort of invariance under permissible transformations in the response scales of each informant. Improved measures of interitem similarity and social network concepts will result from such investigation. (2) How can one assess the quality of the information provided by each informant in order to arrive an an optimal weighting scheme for aggregating responses ? The research develops new statistical models for different scaling techniques such as magnitude estimation, paired comparisons, and the constant sum method. The models will allow researchers to optimize the reliability of aggregated data. The significance of the research is that it will provide new methods of social measurement comparable in quality to those used in such areas as psychophysics, where comparisons with objective physical stimuli are utilized. The interdisciplinary collaboration of the two investigators has been extremely successful in the past. The promise of their ambitious research agenda, both in its conceptual significance and practical import, is equally great.

9309667 BATCHELDER This project will continue research on a class of formal cognitive models known as general processing tree (GPT) models. GPT models are a special class of simple statistical models that have great potential to help researchers measure unobservable cognitive capacities, such as memory storage, memory organization, and retrieval capacity. Prior research involved the invention of a number of new models and their application to different issues in human memory, including bilingualism, eyewitness memory, mnemonics, source memory, and memory loss in the elderly. Also, advances in the statistical theory behind this type of modeling have led to the work beginning to have an impact on the basic research of a number of psychologists. The goal of this new project is to expand this research program and take it to the "next level." This will include the development of new GPT models and expansion of their application to a wider range of topics in psychology. For example, one project will take current complex theories of human cognition and represent them approximately with more simple GPT models. Recent research suggests that the class of cognitive models that can be represented in this manner is quite large. When theoretical models are expressed in GPT form, the extensive statistical theory behind GPT modeling can provide a simple yet thorough method for evaluating and understanding the models. Another project will explore the use of GPT models to measure different types of logical-reasoning capacities, such as the ability to find confirming and disconfirming evidence for a hypothesis. The research will develop GPT models to disentangle these and other factors in reasoning. The result should be a clearer idea of how to assess a person's logical strengths and weaknesses. In addition, the research will include further analysis of the statistical and structural properties of GPT models. This analytic work will include extending the models to han dle individual differences in cognitive abilities and also expanding them to analyze the time it takes to make various responses (i.e., reaction-time data). Finally, a major goal of this project is to develop and document user-friendly software programs on the use of GPT models. The material will be targeted to a large class of research psychologists and will include a general overview of GPT modeling, examples from inside and outside psychology, and the statistical results from this research. The computer programs will be general enough to enable researchers to develop, analyze, and test their own GPT models in a wide variety of situations. These materials will serve to make this methodology more accessible to other psychologists. ***

A significant portion of formal (mathematical) analyses in the social and behavioral sciences during the last half-century has been organized around the modelling of choices and defining an operational concept of rationality, both for individual decision makers and for collective decision makers (i.e., social institutions). This training workshop, to be held at the University of California, Irvine in summer 2000, will provide a detailed introduction to such formal approaches in three substantive areas where the research effort has been intense and shows no sign of abatement, namely: individual decision making in the presence of uncertainty and intertemporality; collective decision making in political institutions; i.e., the democratic sharing of decision power among citizens with conflicting opinions; and fair division of economic and other resources when the recipients have different tastes, needs, or responsibilities. For each area, the theories propose some normative answers (justified by precise axiomatizations and algorithms) and provide some descriptive models (driven by data from the social and behavioral sciences).

The project combines derivations of formal process measurement models with application of the models to substantive empirical research on the universality of the semantic and perceptual domain of color terms and colors. The derivations will extend previous models of cultural consensus theory to two situations for which current models do not exist: (1) continuous response variables, and (2) two or more cultures. Cultural consensus theory is a family of knowledge aggregation models for questionnaire data that permit simultaneous estimates of the culture competence or knowledge of each informant and the consensus correct answer to each question. In addition, the models will incorporate item difficulties as new parameters and provide appropriate estimation procedures. The substantive color experiments will be carried out using the eight basic chromatic colors, and their names, as previously defined by vision researchers. Preliminary results from judged similarities collected in English in the United States and in Chinese in Taiwan showed: (i) robust and reliable individual differences among subjects, (ii) striking similarities between Chinese and English structures, and (iii) the structure of color names is similar to the structure of chromatic colors. Further comparative studies will be carried out in Vietnamese where the linguistic structure for colors is different, e.g., a single color term for English blue and green.

Progress in social science requires the measurement of concepts such as cultural beliefs and social norms that are defined by social conventions. Social concepts are unlike most physical and psychophysical measurement because they have no direct outside criteria for comparison. Because of this social measurements must be inferred from the pattern of responses among informants. This research develops sophisticated measurement models for cultural beliefs and social norms. The application of these models to the perception of colors (and how they are named) in different languages and cultures will lead to understanding the relative importance of individual and cultural variations in how humans perceive the world.

This research project will expand the computational, statistical, and methodological aspects of Cultural Consensus Theory (CCT) to handle a variety of new situations. Knowledge, preferences, and beliefs shared by a group of individuals can provide valuable information to interested parties in a variety of settings. Examples include eyewitness reports of a traumatic event, intelligence reports about the relationships between members of a covert network, grades assigned to student essays by a group of teachers, and shared folk medical beliefs in a particular cultural group. Cultural Consensus Theory (CCT) is a formal, statistical way to analyze responses to questions regarding shared knowledge, to determine if there is evidence for an underlying consensus, and if so, to pool the responses to uncover the shared knowledge within the group. This project will expand CCT to cover examples such as the ones above where there is no available ground truth to determine the answers to the questions that reflect the group consensus apart from the individuals' responses. Estimating consensus knowledge is crucial for science and for addressing important national problems. For example, properly pooled reports from eyewitnesses can assist in police investigations. Proper pooling of intelligence information about a covert network can assist in discovering its nature. Developing better ways to pool the responses of graders can improve the assessment of student ability, and uncovering shared medical beliefs can lead to public health policy that brings beliefs and scientific medical knowledge into proper alignment. Freely available software will be developed for the new models along with user guides.

This research project will increase of the scope of CCT by developing new response models, discovering their properties, and augmenting their statistical inference. CCT models will be developed for a number of different cases: paired-comparisons (where individuals indicate their preferences among pairs of options); networks (where individuals indicate which nodes in the network are connected); and similarity, distance, and triad questionnaires (where, for example, individuals indicate how close or how similar two items are in semantic memory). The new models will be evaluated with statistically generated data and real experimental data.