Robert J. Wirth, Ph.D. - US grants

DESCRIPTION (provided by applicant): The purpose of the proposed project is to explore new methods for testing measurement invariance in latent growth models (LGM). LGMs are a collection of statistical methods with the aim of modeling complex systems of behavior such as substance abuse, AIDS and HIV prevention, functional brain imaging, and academic achievement over time. However, much of the research utilizing LGMs has been conducted by modeling individual sum scores over a series of assessments; although at times appropriate, this technique necessarily assumes that the underlying measurement model remains constant over assessments. That is, it is assumed that the condition of measurement invariance has been retained. Although there is extensive literature discussing the importance of measurement invariance in longitudinal research, little quantitative work has been performed examining the appropriateness of this assumption within the LGM framework. In fact, I am unaware of any empirical research specifically exploring the minimum measurement invariance conditions required to validly apply a LGM. This is critically important because failure to maintain measurement invariance in practice can reduce the internal validity of a study as well as lead to biased or misleading results. Thus, the proposed study will aim to 1) examine the impact of violating the assumption of measurement invariance within LGM; and 2) explore the necessary conditions for the retention of measurement invariance in practice. The results of the proposed project will help insure greater validity in applied substance use research.

DESCRIPTION (provided by applicant): Awareness of the enormous advantages of computerized adaptive testing (CAT) in an item response theory framework (IRT) is growing rapidly in the behavioral research community. CAT has the capacity to significantly reduce respondent burden without losing measurement reliability. Although a small number of computerized adaptive tests (CATs) exist for behavioral measurement, they have been designed as "single-use" CATs which were developed to assess a fixed set of domains. The fact that currently available CATs cannot be easily altered by individuals other than the original developers to incorporate new domains severely limits the usability of the CATs beyond their initial purpose. Thus, despite the growing awareness of this state-of-the-art methodology, it is not currently feasible for the majority of behavioral researchers to implement CAT in their research designs. The construction and maintenance of CATs requires ongoing attention by individuals with extensive psychometric and computing experience. The few available software products for creating CATs are also extremely limited: they are PC-based so necessary software must be deployed on each computer that is going to be used for CAT administration;once software is installed on a PC that PC must be taken to the participants or the participants must be brought to the PC;subsequent software updates require internet connectivity or physical media;and these programs have extensive system requirements. The proposed project aims to overcome these barriers to CAT, making CAT an available tool for any behavioral researcher with access to the internet. [At the completion of Phase II] of the proposed project, a flexible web-based CAT system will be available so that any researcher can log-on to a web-site and create an adaptive test. [As a final goal,] the system will be flexible enough that users can, if desired, create and administer CATs remotely and without VPG assistance. However, researchers wishing to use this system will have the VPG staff available to them to help create and administer CATs, and to incorporate the resulting measures into their research design. The completion of Phase I will result in a prototype web-based CAT system. This [basic] prototype will be designed to receive input from a user through upload of files or direct user entry, incorporate the user input into a CAT algorithm implemented with IRT models, administer an adaptive test, and generate output for direct printing or download by the user. [The focus of Phase I is on the implementation of the core CAT algorithms and basic support input/output systems as dynamic web applications.]Phase II of the proposed project will solicit feedback from a wide array of potential users to expand the prototype, focusing on usability and continued flexibility, and will include pilot testing and refinement to ensure the web-based CAT system and support services are designed to fully meet the needs of behavioral research and assessment communities. PUBLIC HEALTH RELEVANCE: Computerized adaptive testing (CAT) has the capacity to significantly reduce respondent burden without any loss of reliability. However, CAT technology is currently accessible only to those with extensive psychometric knowledge thereby limiting the majority of behavioral researchers from implementing CAT in their own research. The proposed project aims to overcome these barriers to access by creating a flexible web-based CAT system that will allow behavioral researchers to incorporate this state-of-the-art methodology into their study designs.

PROJECT SUMMARY ABSTRACT The proposed Migraine Clinical Outcome Assessment System (MiCOAS) aims to improve and standardize endpoints and the assessment of those endpoints in migraine clinical trials. Such standardized clinical outcome assessments (COAs), developed to incorporate the patient voice and empirically supported by gold-standard instrument development techniques, will aid in the interpretability of results from and across migraine clinical trials and, by extension, advance the development and approval of migraine treatments. The UG3 phase of the grant will first assemble a group of stakeholders (substantive research experts, patient advocates, clinicians, psychometricians, regulators, and payers) to guide the project (AIM 1). Comprehensive systematic review of the migraine literature will be undertaken to fully understand current migraine outcomes, in both acute and preventative migraine trials (AIM 2). Results from qualitative research using patient-centered interviews (AIM 3) will ensure that endpoints identified during AIM 1 and AIM 2 as targets for development/refinement are meaningful to patients. The UH3 grant stage will use additional patient-centered qualitative research to fully understand the patient experience with respect to the target endpoints identified in UG3 and what aspects are most important to them, patient language in discussing said endpoints, and any possible areas of study that were not previously identified (AIM 4). Using the patient-interview results, outcome measures will be developed using gold- standard techniques. In the expected case of developing/refining a preventative quality-of- life/impact/disability-type measure, a new data collection of migraine patients will be designed and executed and gold-standard psychometric techniques will be employed to empirically evaluate and refine candidate items, finalize the items selected to the final assessment, and evaluate the initial validity and fit-for-purpose of the outcome assessment. A separate data collection will also be designed, executed, and analyzed for the draft outcome assessments intended for acute migraine trials (AIM 5). The overarching purpose of this grant is to develop a publicly available core set of migraine endpoints and their related COAs for migraine. Given this, proper dissemination of findings through scientific outlets in the form of presentations and manuscripts is essential (AIM 6). Successful completion of the overall goal of the proposed project will result in standardized and empirically-supported migraine COAs for clinical trials that are meaningful to patients and statistically robust.