Thomas W. Malone - US grants

The purpose of this workshop is to bring together leading researchers from a variety of different disciplines (including computer science, economics and organization theory) that can contribute to the creation of "coordination theory"--a body of principles about how to coordinate the activities of separate actors. One of the intriguing aspects of this interdisciplinary approach is the possibility of exploiting parallels between: (a)coordinating groups of people; (b) coordinating groups of computer processors or program modules, and: (c) coordinating hybrid groups that include both people and computers. The primary activity at this workshop is to explore these issues in more depth, focussing on specific research papers. The workshop will be held February 17-19, 1988 at MIT's Endicott House conference center in Dedham, Massachusetts. A major outcome of the workshop is expected to be an agenda for research and possible funding for this new area.

The goal of this project is to develop knowledge-based environments for supporting group coordination. The PI builds on his earlier experience with the Information Lens (an intelligent system that helps people find, filter, and sort electronic messages) in developing more powerful and flexible tools to support a wide variety of coordination processes. The research builds on a prototype system. The new work will integrate many of the important features of (a) object-oriented databases, (b) hypertext, (c) electronic messaging, and (d) intelligent rule-based agents. The system will allow unsophisticated computer users to create or modify their own cooperative work applications using three key primitives: semistructured objects, customizable folders, and semiautonomous agents. The researchers experiment with a variety of applications in this environment such as: (a) intelligent tracking of tasks for project management, (b) managing design changes in engineering groups, (c) group decision making, and (d) collaborative authoring. In developing these applications, the existing prototype will be augmented with more knowledge of objects, more knowledge of processes, and more kinds of agents. The significance of this work is in making electronic mail systems more flexible and easier to use. It is especially promising for alleviating the burden of managing large electronic mail files by organizing and prioritizing messages.

9318659 Berndt This workshop was held on November 11-13, 1993 in Charleston, South Carolina. It brings together leading individuals from the international research community to explore ways of improving the ability to assess, measure and otherwise understand the economic and organizational impacts of information technology investments. Participants at the workshop come from a variety of different disciplines -- computer science, organization theory and economics, as well as from different research organizations -- academia, national and international statistical organizations, and national research funding centers. Particular attention is devoted to the availability and/or necessity of data sets amenable to testing alternative hypotheses. Another important purpose of the workshop is to set an appropriate research agenda to guide national research and statistical funding agencies and the Organization for Economic Cooperation and Development (OECD). Proceedings of the workshop are to be published as a special edition of the journal, Economics of Innovation and New Technologies, in the Summer of 1994.

*** 9224093 Malone How important are managerial tools like total quality management, business reengineering, and information technology? The goal of this project is to provide a firmer theoretical and empirical foundation for understanding and using them. To do this, the project will include (1) collecting examples of how different organizations perform similar processes, and (2) representing these examples in an on line "process hand book" which includes the relative advantages of the alternatives. The handbook is intended to help (a) redesign existing organizational processes, (b) invent new organizational processes that take advantage of information technology, and perhaps (c) automatically generate software to support organizational processes. A key intellectual contribution of the work is expected to be a novel approach to representing processes at various levels of abstraction. This approach uses ideas from computer science about inheritance and from coordination theory about managing dependencies. Its primary advantage is that it allows users to explicitly represent the similarities (and differences) among related processes and to easily find or generate sensible alternatives for how a given process could be performed. ***

There is a wide gap in current knowledge about the types of conflict, conflict management strategies, and how management strategies are best linked to particular conflicts. Existing knowledge in this area has not been systematized or made formal and computational. This gap has significant practical impact, particularly in the important domain of collaborative design and concurrent engineering. Complex manufactured goods and software are increasingly designed using large-scale collaborative processes. Current, mainly manual, conflict management practices are being overwhelmed by the sheer scale and complexity of modern-day design challenges, leading to severe impacts on cost, timeliness, and effectiveness of product designs. This research will develop, within the MIT Process handbook, a formally represented, publicly accessible repository of conflict management knowledge organized in a way that helps reveal building blocks, key dimensions and regularities, and thereby (2) facilitates important uses such as learning about conflict management and developing general theories of multi-agent conflicts and exceptions. This work makes important intellectual and practical contributions---it will be a key building block for better computational algorithms for managing exceptions (including conflicts) in collaborative work. The repository itself will help researchers carry on a more cumulative ans systematic exploration of the phenomena. http://ccs.mit.edu/klein.html

This is the first year funding of a five-year continuing award. The world of business and organizations is entering a period of dramatic and rapid technology-based transformations that many people believe will be as significant as those that characterized the Industrial Revolution. This project will investigate the profound socioeconomic changes likely to be associated with such transformations through conducting empirical research that is broad-based, in-depth, and longitudinal. Using multiple theoretical and methodological approaches, a panel of strategically-selected firms in established as well as entrepreneurial and emergent businesses will be tracked over time. A comprehensive set of systematic and grounded empirical data in these organizations will be collected and analyzed over five years, and will generate deep insights and general theories about what is really happening as organizations use information technology to transform how they work and interact with the market over time.

The goals of this project are to develop collaborative tools for helping to solve difficult managerial and societal problems and to test these tools on an important problem confronting the world today: global climate change. Inspired by systems like Wikipedia and Linux, the project will develop a global, on-line forum - called the Climate Collaboratorium - in which thousands of people around the world can create, analyze, and ultimately select detailed plans for what humans can do about global climate change. At the core of the system will be an evolving collection of user-created plans based on computer simulations of the actions humans can take and the predicted impacts of those actions. Users will also be able to debate the pros and cons of different plans and vote for the plans and arguments they find most credible and desirable.

Intellectual merit
The primary intellectual contributions of this work will be lessons about how to design large-scale model-centric collective decision-making tools and the communities to use them.

Broader impacts
At a minimum, this on-line forum can help educate the general public about the real issues involved in global climate change. But more importantly, by constructively engaging a broad range of scientists, policy makers, and ordinary citizens, this forum may help develop plans and policies that are actually better than any that would have otherwise been developed. In short, it could become a combination of a kind of simulation game for climate change, a Wikipedia for controversial topics, and an electronic democracy on steroids.

The "holy grail" of artificial intelligence research for decades has been to design computers with robust, integrated, human-like intelligence. This goal has proven elusive, in spite of a massive amount of research. But another goal is just now becoming feasible, and so has been the subject of much less research: using vast computer networks to create new kinds of intelligent entities that combine the best of both human and machine intelligence. One key to designing such human-centered computing systems is better ways of measuring the collective intelligence they exhibit. That is the focus of this research, which represents a collaborative effort among researchers at MIT (lead institution), CMU and Union College. The PIs will first use analogies with what is already known about measuring individual intelligence to suggest new ways of measuring the collective intelligence of complex human-machine systems. For instance, they will determine whether the striking pattern of correlations across tasks that characterizes individual human intelligence even exists for human-machine groups. Next, a series of statistically validated tests will be developed to measure the key components of collective intelligence in human-machine groups. Then, to better understand the "active ingredients" of collective intelligence, the PIs will use what is already known about how groups of people interact effectively to measure micro-level behavior in human-machine groups. A key goal will be to find critical factors (such as group size, technological support, or individual capabilities) that contribute to a human-machine group's adaptability across a wide range of tasks.

Most people and computers today are parts of larger human-machine systems that must cope with a wide range of problems. This research will provide powerful new tools for managing and designing such systems. Imagine, for instance, that one could give a short "collective intelligence test" to a top-management team, a product development team, or a collection of Wikipedia contributors. Imagine that this test could predict the team's future performance on a wide range of important tasks. And imagine that the test could also help suggest changes to the team that would improve its flexibility. Or imagine that designers of new collaboration software tools could use a single test to predict how well their tools would improve a group's effectiveness on many different tasks. From the smallest business work groups to our largest societal challenges, there are now many new opportunities for people and computers to solve problems together, not just more efficiently, but also more intelligently. This work will help build a firmer scientific foundation for doing this.

Broader Impacts: With individual humans, it is relatively easy to measure intelligence, but it is difficult to increase that intelligence or to observe the detailed events inside the brain that give rise to it. With human-computer groups it is much easier to observe and change factors (such as group size, composition, and technological support) that are likely to determine the group's collective intelligence. Thus, there is a profound intellectual opportunity, not just to learn more about how to design intelligent human-computer systems but also to gain new insights into the very nature of intelligence in complex systems. The results of this research, therefore, will be of interest not only to researchers in computer-supported cooperative work, human-computer interaction, and artificial intelligence, but also more broadly to fields such as cognitive science, social psychology, and organization theory.

The rise of Web 2.0 and large-scale distributed contributor systems is generating opportunities for a new interdisciplinary field on the topic of collective intelligence. Like cognitive science did beginning in the 1970's, this new field may help share perspectives and results from a variety of different fields. In the case of collective intelligence, potentially relevant disciplines include computer science, organization theory, social psychology, economics, and others. This workshop will frame a research agenda covering important aspects of collective intelligence such as: how to define and measure collective intelligence, how to motivate participants to form part of an intelligent collective, what is the effect of different patterns of connection among the participants (i.e., network science), what problems are well-suited to be attacked by a collective intelligence approach, and what are common design patterns among successful systems.

The workshop on collective intelligence will facilitate broader impacts in: (a) coalescing a community of people from different disciplines and perspectives, (b) initiating consensus on how to define collective intelligence as a field and the topics that should be central within it, (c) establish a preliminary research agenda for the field, and (d) catalyze publications about the prospects for this field in one or more high quality journals.

The goal of this project is to develop a system that combines the strengths of both human and computer capabilities to help solve a complex societal problem: global climate change. The project will develop an on-line community called the Climate CoLab, in which many thousands of people around the world create, analyze, and ultimately select detailed plans for what humans can do about climate change. At the core of the system will be an evolving collection of user-created proposals based on computer simulations of the actions humans can take and the predicted impacts of those actions. Users will also be able to debate the pros and cons of different proposals and vote for the proposals and arguments they find most credible and desirable. By integrating three capabilities (computer simulation models, on-line debates, and electronic voting) in a novel way, the system lets a very large group of people define and evaluate alternative problem solutions while computers do the rapid calculations needed to assess key consequences of each alternative. By including relevant experts from different disciplines and members of the public who have novel points of view, the community can consider a wide range of plausible alternatives. And by involving policy makers and large numbers of citizens, the eventual political adoption of the most promising alternatives is facilitated.

Intellectual merit: The primary intellectual contributions of this work will be generalizable lessons about how to design large-scale, on-line communities that use computational models to help solve difficult societal and managerial problems.

Broader impacts: The project will help educate the general public about the issues involved in global climate change. In addition, by constructively engaging a broad range of scientists, policy makers, and concerned citizens, this system may help develop plans and policies that are actually better than any that would have otherwise been developed.

From Wikipedia to Linux to scientific and business work-groups all over the world, both online and off-line groups are becoming a pervasive part of modern life. It is becoming increasingly important, therefore, to understand how to improve the performance of these groups. The work proposed here will use a new measure of generalized group effectiveness -- called "collective intelligence" -- to help do this.

Building on previous work by the investigators, the project will first develop an online test for collective intelligence. Then it will compare the results of online and face-to-face groups taking this new test with previous results for groups taking an offline version of the test. This will help clarify the degree to which online and off-line groups differ in their general effectiveness on a wide range of different tasks. Next the project will use this test to systematically measure the collective intelligence of online groups that range in size from 2 to 20 people. This will lay the foundation for exploring whether larger online groups can take advantage of the increased resources that more people bring, without suffering as much from the process losses that usually accompany increased group size in face-to-face groups. Finally, the project will systematically measure the collective intelligence of online groups with varying proportions of women. In doing so, the project will also test one particularly promising explanation for a gender effect on group performance: that groups with more women are less interpersonally competitive, and that this lower intra-group competitiveness leads to higher collective intelligence.

While there have been decades of research on factors that affect the performance of groups, almost all these studies have each focused on a single task. Thus, strictly speaking, the lessons to be learned from this previous work are limited to the specific tasks studied. The work proposed here uses the perspective of collective intelligence to investigate, not just the ability of a group to perform a single task, but the group's general ability to perform a wide range of tasks. Since many real-world groups must cope with a wide range of problems, just such a perspective may be needed to systematically predict their performance. In addition, the approach developed here can provide a significant economy of effort in evaluating potential ways of improving online group effectiveness. Instead of testing interventions on many different specific tasks, researchers will be able to test the interventions once with this general measure, and then have some basis for predicting the effects of the intervention on many other tasks. By making an online test of collective intelligence available to other researchers, the project will help advance scientific practice in this area. More generally, by providing a firmer scientific foundation for measuring and improving the performance of groups, the project may help our society address many of its most important problems more effectively. For instance, with the right kinds of collaboration tools, online groups may be able to be much more effective than face-to-face groups, taking advantage of the simultaneous efforts of far more people without the coordination losses that usually occur in larger groups. And understanding the dynamics of gender diversity may help to improve the collaboration of the groups in which both men and women work, by giving everyone's best ideas a better chance to be heard. And perhaps, someday, this will help create groups that are more collectively intelligent than any groups have ever been before.

From writing an encyclopedia, to developing software, to folding proteins, more and more problems that used to be solved by small groups of experts are now being solved by much larger "crowds" of people using the Internet. For many complex problems, however, the creativity, energy, and diversity that crowds bring are not enough to solve the problems. Various kinds of specialized knowledge that only experts have are also needed. This project focuses on creating the cyber-infrastructure to combine these two kinds of resources--experts and crowds--in solving complex societal problems. The project will address this question in the context of what many people believe is one of humanity?s most important sustainability challenges today: how to deal with trending environmental dynamics. Much of the proposed work includes developing and testing ideas on the team's CoLab, a software platform and online community focused on this issue, which was developed with prior NSF and other support. More specifically, the project will investigate how experts and crowds can work together to perform four key problem-solving activities: decomposing the overall problem into smaller pieces, generating potential solutions for the pieces, integrating the pieces into overall solutions, and evaluating the solutions. For example, the project will study (a) whether crowds generate better ideas when they start with "seed" ideas from experts, (b) how certain kinds of expert knowledge can be encoded in computer simulation models that crowds can use on their own, and (c) to what degree expert evaluations can be approximated by combinations of semi-experts, novices, and software tools.

The results are expected to include (a) open source software applicable to many sustainability and other challenges, (b) a set of processes and methodologies with which communities can effectively use this software, and (c) a diverse community of tens of thousands of people using this approach to address issues associated with trending environmental changes. The primary intellectual contribution of the proposed work will be better processes and computer tools for creating on-line communities that combine the best features of both experts and crowds to solve complex societal problems. Many aspects of the technical and organizational approaches to be studied should be of interest to researchers in a variety of fields, including collective intelligence, computer-supported cooperative work, human computer interaction, computer science, engineering, organizational design, psychology, and public policy. By engaging students and the general public to come up with credible ideas for what people can do about complex problems, this project will help educate a much broader community about the actual issues involved. If successful, the work will likely lead to the development of better approaches for complex societal problems than any that would otherwise have been developed. Many aspects of the results will be applicable to a wide range of domains such as education, healthcare, and business problems like strategic planning and budgeting.