Bing Liu - US grants

Text classification is the automated assignment of text documents to pre-defined categories/classes. Traditionally, a classifier is built using training documents labeled (often manually) with pre-defined classes. This classifier is then used to classify new documents into those classes. This classic model is called supervised classification because the training documents all have pre-labeled classes. Although this model is important, in practice it may not be applicable in some common situations. For example, given a set of documents of a particular class P (positive class) for training, one may want to classify a set M of mixed (unlabeled) documents that contains documents from class P along with other types of documents (negative documents) into documents from P and documents not from P. Since there are no labeled negative documents, the traditional classification techniques are inapplicable. This problem is called partially supervised classification (PSC). Existing methods for solving PSC are based on heuristics, which are prone to error. Researchers typically use the evaluation method for supervised classification to evaluate PSC techniques, which is inadequate because supervised classification approaches assume the availability of labeled negative documents.

The objectives of this project are to design a robust and principled technique to solve PSC, implement a system for PSC, devise a method to evaluate such techniques, and identify methods for determining the minimum number of labeled documents needed to achieve the optimal accuracy in order to reduce manual labeling efforts. The algorithms and systems developed by this project will be used in a graduate course and put on the Web for other researchers to use in their research and teaching. The results of this research should be widely useful because the identification of targeted information/documents is of great value in this information age.

Social science takes as axiomatic that agents act on beliefs: however, teasing out the correlations between belief and action is no simple feat. Traditionally, social scientists relied on survey data to measure sentiment that might indicate changing economic trends, and on lab experiments to test actions given manipulated beliefs. Surveys, however, are notoriously expensive to scale, difficult to conduct frequently, and possess bias; lab experiments artificially constrain decision-making and thus fail to capture the complex dependencies of real-world actions.

This project will generate measurements of public sentiment directly from the actions of people, using one of the largest datasets of human behavior ever studied: online opinions expressed on platforms like Twitter or comments posted on news websites, time series representing the volume of search queries on Google, and call detail records. Using these digital footprints, the project will develop new social-computational measures of public sentiments related to the state of the economy, expected unemployment, and concerns about national priorities.

Broader impacts: The project will offer new alternatives to surveys as a measure of public sentiment, and will also generate unprecedented insight into the online and onsite behavior of the American population. This research also has the potential to help public and private organizations better understand the dynamic behaviors of customers and constituents, and to make business and policy decisions informed by economic trends derived from data. The project will enhance education through the interdisciplinary training of graduate students. Both graduate and undergraduate students from underrepresented groups will be actively encouraged to participate in the project. A public website will also be set up detailing the project results.

Given user reviews on Web sites such as Yelp, Amazon, and TripAdvisor, which ones should one trust? Online reviews have become an important resource for public opinion sharing. They influence our decisions over an extremely wide spectrum of daily and professional activities: e.g., where to eat, where to stay, which products to purchase, which doctors to see, which books to read, which universities to attend, and so on. However, the credibility and trustworthiness of online reviews are at stake. It is well known that a large body of reviews is fabricated -- either by owners, competitors, or entities paid by those -- to create false perception on the actual quality of the products and services. What is more, opinion fraud is prevalent; while credit card fraud is as rare as 0.2% or less, it is estimated that 20-30% of the reviews on well-known service sites could be fake. This poses a serious risk to businesses and the public, from investing on a low-quality product to consulting an incompetent doctor for diagnosis and treatment. Like other kinds of fraud, opinion fraud is a serious legal offense. In fact, it is currently being recognized as a serious issue in law enforcement by policymakers. Thus solving this problem is of great importance to businesses and the general public alike. Accurately spotting opinion fraud will enable site owners to provide trustworthy content, maintain the integrity of their service, and protect the online citizens from unfair (or potentially harmful) products and services. Businesses will also benefit from reviews with reliable feedback. Honest businesses will be indirectly rewarded, as it will no longer be easy for unscrupulous businesses to benefit from fake reviews. The research outcomes will thus contribute significantly to the healthy growth of the Internet commerce. Educational activities include incorporating research findings in graduate level courses, educating public on fraudulent behavior and misinformation, and providing publicly available educational materials including lectures and manuscripts.

Given the critical issues of opinion fraud in online communities, how can one identify fake reviews and attribute responsible culprits behind them? By conjoining expertise of the PIs over various modalities of deception footprints ranging over language, user behavior, and relational information, this project presents a research program that will result in much needed solutions to this emergent, prevalent, and socially impactful problem. The ultimate goal is to create a unified detection framework via synergistic integration of multiple information sources; from linguistics, user behavior, and network effects, to obtain the best of all worlds. The main idea is to formulate the problem as a relational inference task on composite heterogeneous networks, providing a principled, extensible approach that can blend and reinforce all the above cues towards effective and robust detection of fraud. From a scientific point of view, the research brings together three disciplines: natural language analysis, behavioral modeling, and graph mining. The outcome is a suite of novel, principled, and scalable techniques and models that will enhance our understanding of the creation and dissemination of opinion fraud and misinformation in general at a large scale. The PIs will collaborate with industry partners such as Yelp, Google, and Amazon, directly solicit online fake reviews, and conduct well-designed user studies for testing and validation of their techniques. The project web site (http://www.cs.stonybrook.edu/~leman/PROJECTS/OPINION_FRAUD/) provides additional information and will include open-source software and datasets.

Health coaching is an effective process for improving poor health behaviors by providing education on health-related topics, setting personalized and realizable health-related goals, monitoring and encouraging progress towards those goals, and sequencing or refining a progression of health goals over time. Though useful, its highly personalized and labor-intensive nature makes the cost of effective health coaching prohibitive for many underserved populations that could benefit significantly from it. This EArly Grant for Exploratory Research (EAGER) seeks to conduct an exploratory investigation of the technical feasibility of creating a virtual health coaching system that learns from expert demonstration to interact with patients via the smart message system (SMS). The project develops two key initial components: a health goal summarization component that extracts details of proposed and agreed upon health goals from SMS dialogues between participant and health coach; and a health goal assessment component that estimates the suitability of proposed health goals in terms of the five dimensions of goal setting used by the coaching experts: Specificity, Measurability, Attainability, Relevance, and Timeliness (SMART).

This EAGER project develops techniques to interpret short, telegraphic, and often ungrammatical SMS messages and to extract details of health goals for improving physical activities established in those messages using natural language processing and structured prediction methods. It expands sentiment analysis methods to identify both noun and non-noun targets of emotion and uses these analyses in combination with inverse optimal control methods to assess the suitability of the sequence of proposed health goals over the course of the dialogue in terms of each of the five SMART dimensions. Evaluation of the developed methods is planned using a collected corpus of SMS-based communications between a health coach and participants annotated with tags relating to semantics, sentiments, and health goals. Successful development of these capabilities represent an important first step for realizing a virtual health coaching system that is able to provide personalized health coaching benefits of comparable quality to a human health coach.

Health coaching is an effective process for improving health behaviors by providing education on health-related topics, setting personalized and realizable health-related goals, monitoring and encouraging progress towards those goals, and sequencing or refining a progression of health goals over time. Though useful, its highly personalized and labor-intensive nature makes the cost of effective health coaching prohibitive for many underserved populations that could benefit significantly from it. This project will develop a virtual health coaching system that learns from human health coach demonstrations to interact with patients via the smart message system (SMS) to set appropriate health goals that are Specific, Measurable, Attainable, Relevant, and Timely (SMART). The virtual coach will become progressively more independent, but will not be fully autonomous for two reasons: first, at this stage of research, one cannot trust the system to always send appropriate messages when it encounters unique situations; second, the virtual coach would not substitute, but supplement/collaborate with a human coach. Successful development of these capabilities represents an important step towards more scalable health coaching systems that do not sacrifice the effectiveness realized by a human health coach.

This project develops the two key components of a virtual health coaching system: a dialogue management and sentiment analysis component that guides and analyzes communication with users; and an imitation learning component for learning high-level SMART goal-setting and coaching strategies from the human health coach's interactions with users. Significant technical advances that address the underlying problems from these sub-areas are needed to achieve human-level health coaching efficacy and positive health outcome rates. These include: novel natural language processing methods to process informal, concise SMS including the emotion and mood they convey; expression of increasing autonomy via linguistic proxies such as initiative; and inverse optimal control methods for rationalizing observed behavior. Building on an initial feasibility study, this project will assess the benefits of a developed virtual health coach with varying degrees of autonomy for SMART goal setting and additional health coaching for a diverse range of users through SMS and fitness tracking devices.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Sentiment analysis is the task of extracting and classifying people?s sentiments, opinions, and emotions expressed in social media and other sources. It has a very wide range of applications because consumer and public opinions, sentiment expressions are instrumental for decision making of businesses, organizations, and individuals alike. Although sentiment analysis has been investigated extensively in the past, its accuracy is still low, which limits its scope of applications. In the past few years, we and some other researchers used a new machine learning paradigm, called lifelong learning, to help solve some sub-problems of sentiment analysis with promising results. Lifelong learning aims to imitate human learning by learning continuously, retaining/accumulating the knowledge learned in the past, and using or transferring the past knowledge to help new task learning and problem solving. In the process, the learner becomes more and more knowledgeable and better and better at learning. Traditional machine learning learns in isolation, and it uses only the data of the particular application to learn a model. This research will design more effective principled lifelong learning algorithms for sentiment analysis and to create a continually updating knowledge base of attributes (or aspects), e.g., for electronic products, and opinions about them and develop a single framework to significantly boost the performance of sentiment analysis. To broaden the impacts of this project, mature technologies will be transferred to industry and demonstrated and used in classes, and students will participate in the research.

Specifically, the project will develop novel lifelong learning algorithms for four core sub-problems of sentiment analysis with the goal to improve: (1) aspect extraction through learning on the job, which learns to improve the model while working after model building; (2) aspect grouping through lifelong aspect topic modeling, which uses past grouping of aspects to help new grouping; (3) aspect sentiment classification using lifelong attention models, which retain the previous attention distributions and leverage them to build more accurate sentiment classifiers for new tasks; and (4) coreference resolution via continuous association learning to discover aspect and sentiment related coreference relations. The resulting algorithms will be incorporated into a holistic lifelong heterogeneous-task learning model to significantly improve sentiment analysis accuracy. Beyond sentiment analysis, the project will also develop general lifelong learning algorithms that can be applied to a wide range of other applications based on insights gained from lifelong sentiment analysis. For dissemination, in addition to publishing research papers, we will organize workshops and give conference tutorials on the project topic, and release our annotated data and implemented software to the research community.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.