Li Cai, Ph.D. - US grants

DESCRIPTION (provided by applicant): With an increased recognition of the chronic relapsing nature of drug addiction, national efforts are being made to foster the redesign and development of recovery-oriented system of care in order to support sustained recovery. There is a clear need to develop a comprehensive measurement tool that captures the multidimensionality of recovery. Meanwhile, awareness of the enormous advantages of Item Response Theory (IRT) is growing rapidly in the behavioral research community. The goal of this proposed project is to use IRT to help develop, evaluate, and provide initial validity evidence for a measure of recovery. To achieve this goal, we will conduct secondary analyses on data collected in the national Drug Abuse Treatment Outcome Studies (i.e., DATOS that includes adult patients, and DATOS-A that includes adolescent patients) which contain a comprehensive set of measures pertinent to recovery. The project is organized around three specific aims. We will apply IRT methods to screen and identify potential items that can enhance the measurement of recovery from the existing pool of measures in DATOS (Aim 1). We will investigate potential measurement non- invariance in the developed recovery scale across subgroups defined by gender or age (Aim 2). We will also obtain initial validity evidence of the developed recovery scale via longitudinal data analysis (Aim 3). The application of the state-of-the-art IRT methodology will lead to a recovery measure that is psychometrically sound, forming a solid foundation that can be continuously improved upon. In addition, the application of IRT to solve measurement problems in addiction research is not only substantively innovative, but would lead to significant opportunities for methodological advancements as well. Taken together, we believe the proposed study has the potential for making significant unique contributions to drug addiction research, treatment and evaluation, and to the IRT methodology in behavioral measurement. PUBLIC HEALTH RELEVANCE: A sound measure of recovery from drug addiction has important implications for how treatment systems should be structured, delivered, and evaluated. This proposed research will develop an improved measure of recovery, and will gather initial validity evidence of the developed recovery measure. The improved measure is relevant to public health as it will improve monitoring and evaluation of substance abuse-related services designed to initiate and promote recovery from addiction.

Cognitive diagnosis models have received increasing attention within educational and psychological measurement. The popularity of these models largely may be due to their perceived ability to provide useful information concerning both examinees and test items. However, the validity of such information may be undermined when diagnostic models are misspecified. This project focuses on one aspect of model misspecification: violations of the local item independence assumption. It examines potential causes and consequences of such dependence, with particular attention to those causes unrelated to the attributes a diagnostic test is intended to measure. The project proposes and evaluates a hierarchical diagnosis model as an alternative to traditional diagnosis models in which nuisance dependence is ignored. This model maintains the desirable properties of existing models while allowing for greater complexity in the underlying response process. Importantly, the model may be estimated efficiently, even for models with a large number of nuisance latent variables, using an analytical dimension reduction technique described by Gibbons and Hedeker (1992).

There is growing interest in extracting model-based diagnostic information from assessments in order to provide more useful feedback to stakeholders. Up to this point, however, the question of whether traditional cognitive diagnosis models fit real test data has been somewhat neglected. This project examines the issue of model fit and presents a model that may better account for certain causes of misfit than the traditional diagnosis models. To the extent that the proposed framework better accounts for the structure of real test data, its application will contribute to improved test development and lead to more valid model-based diagnostic inferences, such as the classification of test takers according to cognitive attributes or skills. This, in turn, is expected to enhance decision making, as better diagnostic information may allow for more effective (or better targeted) delivery of instructional strategies or clinical interventions. The results from this project will benefit the psychometric practice in any social and behavior science discipline that involves testing and measurement. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career.