Künstl Intell (2013) 27:45–52DOI 10.1007/s13218-012-0231-2

R E S E A R C H P RO J E C T

Solving Wicked Social Problems with Socio-computationalSystems

Joshua Introne · Robert Laubacher · Gary Olson ·Thomas Malone

Published online: 28 December 2012© Springer-Verlag Berlin Heidelberg 2012

Abstract Global climate change is one of the most chal-lenging problems humanity has ever faced. Fortunately,a new way of solving large, complex problems has be-come possible in just the last decade or so. Examples likeWikipedia and Linux illustrate how the work of thousandsof people can be combined in ways that would have beenimpossible only a few years ago. Inspired by systems likethese, we developed the Climate CoLab—a global, on-lineplatform in which thousands of people around the worldwork together to create, analyze, and ultimately select de-tailed plans for what we humans can do about global climatechange.

The Climate CoLab has been operating since November2009, and has an active community of thousands of users. Inthis article, we outline some of the challenges faced in de-veloping the system, describe our current solutions to theseproblems, and report on our experiences.

Keywords Collective intelligence · Collaborativeplanning · Climate change

Portions of the introductory text of this paper appeared previouslyin [1].

J. Introne (�) · R. Laubacher · T. MaloneCenter for Collective Intelligence, Massachusetts Instituteof Technology, Cambridge, USAe-mail: jintrone@mit.edu

R. Laubachere-mail: rjl@mit.edu

T. Malonee-mail: malone@mit.edu

G. OlsonDonald Bren School of Information and Computer Science,University of California at Irvine, Irvine, USAe-mail: gary.olson@uci.edu

1 Introduction

Many important decision-making problems in the real worldare so-called “wicked problems”—problems for which nosingle computational formulation of the problem is suffi-cient, for which different stakeholders do not even agree onwhat the problem really is, and for which there are no rightor wrong answers, only answers that are better or worsefrom different points of view (see, e.g., [2–4]). For exam-ple, most social problems (including the environment, healthcare, poverty, education, and crime) are wicked in this sense,as are many problems in management (including strategicdecision-making and product design).

The problem of global climate change is a wicked prob-lem. In fact, many people would say it has some character-istics that make it especially challenging (or “super-wicked”[5]): Time is running out, there is no central authority thatcan implement a solution, and it is truly universal: it affectsevery one of us and is affected by all of our actions.

Left to their own devices, scientists, journalists, politi-cians, businesses, and consumers will ultimately do some-thing about this problem. But the inefficiencies, delays,and distortions of traditional mass media, political decision-making, markets, and scientific publication mean that the re-sults will almost certainly not be as good as we might hope.Fortunately, however, in just the last decade or so, a new wayof solving global problems has become possible. Exampleslike Wikipedia and Linux illustrate that it is now possible tocombine the work of thousands of people in ways that wouldhave been impossible only a few years ago. Inspired by sys-tems like these, we developed the Climate CoLab—a global,on-line platform in which thousands of people around theworld work together to create, analyze, and ultimately se-lect detailed plans for what we humans can do about globalclimate change.

46 Künstl Intell (2013) 27:45–52

Since the CoLab was first launched to the public inNovember of 2009, it has attracted a diverse community ofthousands of users from around the world who have used theplatform to generate dozens of candidate solutions to differ-ent parts of the problem. In this article, we describe the chal-lenges we have faced in developing the system, our currentsolutions to them, and report on our experiences.

2 Background

When people involved in making group decisions can onlyuse communication tools like face-to-face meetings, tele-phones, and paper-based communications, it is very difficultto have more than a few people deeply engaged in analy-sis and decision-making. Even with modestly sized groupsof people, groups may experience a range of inefficien-cies and process losses [6]. Group decision support systems(GDSS) have sought to grapple with such problems [7].But the size, complexity and open-endedness of decision-making for large-scale social problems such as global cli-mate change far outstrips the capabilities of traditional tech-nological solutions.

A new generation of social-computational systems (likeWikipedia, Linux, and InnoCentive) may be able to pickup where decision support technologies have left off. Thesesystems leverage the combined efforts of very large groupsof people to solve complex problems and create large-scaleproducts. Enabled by cheap, fast access to the Internet, theseare often referred to as “collective intelligence” systems [8].

There are many examples of such systems, but there isno clear recipe for their development. Creating a collec-tive intelligence platform to solve global climate change re-quires novel solutions to a range of socio-technical designproblems. In our initial platform development, three suchchallenges have been paramount: breaking the problem intopieces and subsequently recombining solutions, predictingthe impacts of proposed solutions, selecting good solutions.Each of these challenges, and our current solutions to them,are presented below.

2.1 Designing the CoLab: challenges and solutions

The Climate CoLab is a publicly accessible website (http://climatecolab.org) where members collaborate to createproposals during contests that address aspects of climatechange. Proposals are similar to wiki articles in that theycan be edited on the site and changes reverted. There is alsoa dedicated page for conversation about each proposal. Pro-posal authors have the ability to configure whether any com-munity member can edit the proposal or if editing privilegesare available by invitation only. For some kinds of propos-als, it is possible to attach a model run that supports claimsmade in the body of the proposal.

In addition to writing proposals, some members of thecommunity play special organizational roles. A team ofModerators monitor the site for spam and inflammatory oroff-topic content. Fellows are students and concerned cit-izens who encourage and moderate contest activity. Advi-sors are experienced professionals who frame the focus ofcontests, provide input on proposals, and help to bring con-test results to key stakeholders. Finally, the Expert AdvisoryBoard and broader Expert Council, which include some ofthe most respected climate change researchers in the world(see [9] for a list of current members), provide general ad-vice about the project and review specific content on the site.

We have faced a variety of design challenges in devel-oping the CoLab. One challenge is how to break the prob-lem down into manageable pieces and subsequently weavethe solution back together. Others have focused upon aspectsof this problem in relatively constrained, well-behaved do-mains (e.g. [10–12]), yet the domain of climate change isvast, dynamic, heavily interconnected across different di-mensions, and highly uncertain. For instance, one dimensionof the problem concerns energy production from fossil fuelsand the resulting emission of carbon dioxide. Yet these con-cerns cut across geo-political boundaries and are also con-strained by social and economic factors, all of which willchange in an uncertain manner over time.

We have sought to use existing knowledge to develop aninitial decomposition for the climate change problem. Theclimate change community at large has over time convergedupon a functional decomposition of the climate changeproblem, and this is reflected in the structure of its variousworking groups and reports. We have sought to codify thisstructure in a taxonomy, and to use this taxonomy to focusthe CoLab members on sub-problems. In developing the ini-tial draft of the taxonomy, the Climate CoLab team reliedgreatly on the Fourth Assessment Report of the Intergov-ernmental Panel on Climate Change (IPCC), especially thecontributions of Working Group 3 on mitigation [13] andWorking Group 2 on adaptation [14]. The team then circu-lated the draft taxonomy to multiple experts, and revised itbased on their input.

The current version of our taxonomy defines three dimen-sions:

• What action is being taken—e.g. improving home ef-ficiency, decarbonizing electric power, persuading Con-gress to pass climate mitigation policy, etc.

• Where the action is being focused—e.g. in developedcountries, in the United States, in China, etc.

• Who is taking the action—e.g. individual citizens, utili-ties, any organizations or individuals, etc.

Each of these dimensions is further specified with oneor more hierarchies. Figure 1 shows a partial view of the“What” taxonomy; the complete taxonomy can be found at[15].

Künstl Intell (2013) 27:45–52 47

The taxonomy defines a map of the climate changeproblem space; one might envision the space as a three-dimensional matrix (with who, what, and where as axes).Entries in each dimension of the taxonomy are used to spec-ify regions of the matrix. Not all addressable regions withinthis space present problems worth solving (e.g. it might notmake sense to ask how church groups can help metro re-gions cool the ocean to offset climate driven temperature in-creases), but we believe most problems worth solving canbe located within the space. Because dimensions are or-ganized hierarchically, the taxonomy contains initial infor-mation about how sub-solutions can be integrated to coverlarger portions of the problem space.

– What is the impact of the actions on physical systems (en-gineering, biological, and earth systems)+ Reduce greenhouse gas (GHG) emissions (Mitigation)+ Adapt to climate change (Adaptation)+ Reduce the warming effects of GHG emissions

(Geoengineering)– How does the action affect physical systems?

+ Physical actions: Technologies and practices that ad-dress climate change through their direct impact onphysical systems

+ Social actions: Political, economic, behavioral, or otheractions that influence people to take physical actionsthat address climate change

Fig. 1 Partial view of the “what” dimension of the taxonomy. Theview is expanded to the second level

The taxonomy is used in two ways in the CoLab. First,each contest (and its associated proposals) is located withina region of the problem space defined by the taxonomy (seeFig. 2). As the site accumulates content, this will serve as auseful way to index and search through solutions developedby the community.

The taxonomy can also be used to specify relationshipsbetween contests on the site and guide solution integration.For example, in the currently active contest cycle, an initialset of contests about focused regions of the problem spaceis underway. After these contests end, we will launch con-tests that focus on larger regions of the problem space thatcontain the areas of the problem space explored in the ini-tial contests, and community members will then be invitedto build integrated proposals leveraging work done in theinitial contests.

Another challenge is to predict what will happen if a pro-posal is implemented. Computer simulations that make suchpredictions have served as the cornerstone of policy discus-sions about climate change, but access to these models ismediated by the experts who run them and interpret theirresults. This arrangement puts a bottleneck between stake-holders and the results they require to make decisions. Forexample, this bottleneck makes it much more difficult for agroup to explore a diverse set of planning possibilities with-out active participation of the modeling experts at each stepof the way.

Other fields in which modeling and simulation are heav-ily used for policy making have begun to develop systemsthat incorporate access to a model in order to eliminate thisbottleneck (e.g. [16, 17]). Yet the field of integrated climate

Fig. 2 The active contest index. Note that each contest is associated with a region of the problem space using the taxonomy

48 Künstl Intell (2013) 27:45–52

Fig. 3 The CoLab modeling interface

assessment models, the primary type of tool used to informpolicy deliberations about climate change, is large and no-table for its divergent assumptions and often conflicting re-sults. Providing non-technical end users with comprehensiveaccess to these models requires an approach for synthesiz-ing results from multiple models and communicating effec-tively about uncertainty and the sources of divergence be-tween models. There are also social and organizational hur-dles to enlisting the help of model authors, which is requiredif their models are to be made publicly available.

To facilitate a solution to this problem, we have created aweb-service called ROMA (Radically Open Modeling Ar-chitecture) that wraps existing simulation models (hostedeither externally or internally) in a common API and pub-lishes a web-accessible endpoint for running them [18]. Inaddition to providing a layer of abstraction around existingmodels, ROMA can transform Microsoft Excel spreadsheetsinto runnable simulations and also connect models to createcomposite models.

The CoLab generates user interfaces for models hostedby ROMA (see Fig. 3). Proposal writers use these models tovalidate claims about their proposals. One integrated model,which is a composite of multiple models that are connectedby ROMA, is currently available to the CoLab community.This model takes as input a set of projected changes in fossilfuel emissions, disaggregated by region, and global defor-estation/aforestation, and predicts changes in atmosphericcarbon dioxide concentration, global temperature, sea level,

as well as economic costs, economic damages of climatechange, and negative qualitative impacts to human and phys-ical systems such as agriculture and health. More informa-tion about the specific models used can be found on the Cli-mate CoLab web site [19].

For some kinds of proposals, the existing simulationmodel cannot provide useful projections of future impacts(e.g. estimating the impact of geo-engineering on globaltemperature change). In these cases experts are asked to vali-date claims about impacts. In the future, we hope to continueadding models to support a broader range of proposals, andhave engaged leaders in the modeling community to do justthis.

A third challenge concerns how to focus the community’sattention on the most promising proposals being exploredand, ultimately, how to select the best ones. The simplestway of doing this is with various forms of on-line voting andrating (e.g., Reddit, Amazon, Netflix). Sophisticated ver-sions of this include pairwise voting [20], preferential vot-ing [21], and range voting [22]. But pooling the opinions ofmany only yeilds good results when the participants have theexpertise required to make informed judgments [23]. Thus,our challenge has been to enable collective decision mak-ing that incorporates the expertise required to identify good,feasible solutions.

Our current approach to this problem is to engage a panelof expert judges to select a set of viable alternative proposals

Künstl Intell (2013) 27:45–52 49

Fig. 4 Membership growth andsite visitors over time (Colorfigure online)

in each contest that community members then vote upon.Contests unfold as follows:

1. Community members create proposals within the contextof a contest, which poses a question about a specific as-pect of the climate change problem.

2. At the end of this first phase, domain experts evaluateproposals for feasibility, novelty, likely impact on climatechange, and presentation quality, and select a set of final-ists to move on to the second phase. Judges also providecomments and critiques for each of the finalists.

3. Proposers then have a period of time to improve their pro-posals, after which the community and judges vote to se-lect the most desirable ones.

Awards are given for both those proposals that receive themost votes, and those selected by the judges. This strategyhas successfully increased the feasibility and quality of pro-posals selected by the community [1].

3 Experience

We have been collecting standard web analytics data viaGoogle Analytics since 12/07/09. As of Sep. 18, 2012, theCoLab had 43,460 unique visitors from 175 countries. Ofthese, 3,895 (9 %) have registered for the site. Figure 4 de-picts site growth and visits per day.

Prior to 2012, contests were organized one or two at atime on an annual basis. Large outreach efforts were under-taken to build participation for these contests, using both so-cial media and traditional channels. Since the site launchedthree such contest cycles have been completed. Site trafficand membership growth have been heavily influenced bythese cycles, with the voting period of each contest driv-ing a large number of visitors and new registrations. Thuscontests have served as a valuable mechanism for buildinginterest and membership.

Closer examination of site activity demonstrates that theinitial contests contributed to membership growth, but therewas relatively little substantive activity between contests(Fig. 5). As the member base has grown though, we havebegun to see evidence of ongoing site activity that is notclosely tied to the contest schedule. Members have contin-ued to develop proposals, and have had ongoing discussionsabout the site design, the taxonomy, and new contest topics.We interpret this as evidence that the community is becom-ing self-sustaining.

3.1 CoLab contest results

Table 1 includes a summary of the major contest cycles todate. The first contest began with the official launch of theClimate CoLab in November 2009. This contest had a singlephase and there was no expert involvement. By the end ofthe contest, 162 users had registered on the site, and 58 hadvoted on a plan. The plan receiving the most votes specifiedemissions reductions in all regions of the world by 99 %by the year 2050. All the experts we consulted with saidthat it would not be feasible to make these reductions (for arange of social, economic and political reasons), and yet themajority of CoLab voters selected it.

Because of this result, we introduced the phased contestcycle described above. Proposals generated in subsequentcontests have in general been substantially more compre-hensive and of higher quality than those in the first contest.For example, in the first contest, the popular choice plancontained no information about how the proposed emissionscommitments might be achieved. In contrast, the popularchoice plan from the second contest contained ten times asmany words and provided both rationale and a range of sug-gested actions for implementing the plan.

Generally speaking, winning proposals have been di-verse, creative, serious contributions from people all over

50 Künstl Intell (2013) 27:45–52

Fig. 5 Categorized site activity. The y-axis reflects individual actionstracked by the system, such as posting comments, voting, and editingproposals. The “Proposal” category captures all work on proposalsincluding text edits, updating model runs, and managing teams. Data

is binned weekly, and the y-axis is capped at 140 total actions in orderto see activity detail (activity spikes to >2000 actions during the thirdcontest cycle) (Color figure online)

Table 1 Summary of major contest cycles to date

Dates Contests Phase I proposals Finalists Votes cast

11/1/2009–12/9/2009 1 20 n.a. 58

10/1/2010–11/26/2010 1 29 4 403

5/16/2011–1/15/2011 2 84 12 2100

8/05/2012– 6 26 – –

the world. A complete analysis is not possible here, but asummary of proposal finalists from the 2011 contests (Ta-ble 2) offers a flavor of the kinds of solutions the CoLabcommunity has generated.

4 Conclusion

The Climate CoLab is a novel collective intelligence systemdesigned to help thousands of people around the world worktogether to solve the problem of global climate change. Ithas attracted a continuing stream of visitors and membersfrom around the world. As the platform and its communityhave evolved, members have generated proposals of sub-stantial novelty and increasing quality. The community isnow beginning to collaborate on the site outside of our an-nual contest cycles. These results are evidence that we are onour way to overcoming a key hurdle in the creation of a col-lective intelligence platform—the formation of a large anddiverse community collectively engaged in solving a singleproblem.

The community members who write proposals and en-gage in discussions on the site are by far the most visiblecontributors to the Climate CoLab, and are a critical com-ponent of the system. But the full scope of human intel-ligence woven into the CoLab is far greater. Each of the

design solutions described here is heavily dependent uponother human systems. The domain taxonomy codifies struc-tures that the climate change community has generated in itseffort to grapple with climate change. Radically open mod-eling makes it possible for end-users to leverage the exper-tise of many different model developers, and we rely upon avibrant modeling and simulation community to provide theembedded technology. Expert-mediated voting is criticallydependent upon the energy and willingness of experts whovolunteer their time.

We believe the Climate CoLab is representative of ageneral approach to melding human intelligence and socialtechnology to solve wicked social problems. It is a socio-technical system writ large, that leverages not only the in-telligence of thousands of community members, but also theknowledge and capabilities of many pre-existing human sys-tems. The platform itself is merely a nexus in which we hopeour vast potential collective intelligence may be applied tosolve the problem of climate change.

Acknowledgements We would especially like to thank John Ster-man and Hal Abelson of MIT for their support and participation inmany phases of this project. We would also like to thank the followingfor financial support of this project: the National Science Foundation,BT plc, Cisco Systems, Argosy Foundation, the MIT Energy Initiative,and the MIT Sloan Sustainability Initiative. In addition, we are grate-ful to Stuart Scott and the members of the Climate Summit for theirparticipation in the early outreach efforts, Mark Klein for his advice

Künstl Intell (2013) 27:45–52 51

Table 2 Proposal finalists from the 2011 Contest Cycle. The two contests focused on the economy at the global and national levels, respectively.The “proposal pitch” is a brief description provided by authors to describe their proposals

Proposal title Proposal pitch Author description Awards

Global Proposals (Finalists)

2010 Winners combined No pitch provided Collaboration of winnersfrom prior year

Popular Choice

The Planet or your Plate Mitigate climate change by a rapidreduction of the short lived warminggases by advocating for less meatconsumption globally

1 activist (US),1 professor (US),1 scientist (Australia)

PopularChoice/Judges’Commendation

Rewire Plus: Behaviorchange and value changefor the emerging greeneconomy

A shift to a green economy will requirechanges in behaviors and values all theway down to the individual. Here’s howwe get started!

Sustainability director,Univ. of Toronto; Recentgraduate Univ. Toronto;CEO of SafaraSustainability Solutions

National Proposals (Finalists)

Cycling Carbon A pragmatic, ambitious 7-point plan torenew the U.S. economy, enrich itscitizens, and lead the world in fixing theclimate

Software engineer fromNorth Carolina

Popular Choice

Dream for a GreenFuture

India and dream for a green future : It’stime to make it a reality...

2 University Students atTERI University, India

Popular Choice

Personal Rapid Transitgrids

Install connected Personal Rapid Transitgrids over the urban and suburban areasthat house the densest 50 % of the USpopulation

Research scientist (MIT) Judges’ Choice

Climate proofing theeconomics of sociallysustainable small-scaleagricultural systems

This proposal climate proofs theeconomics of sustainable agriculture inAfrica

3 employees of theNational Center forTechnology, Nigeria/1 Professor, ObafemiAwolowo University,Nigeria

Judges’ Choice

How to Change USEnergy in One GrowingSeason

How to Change US Energy in OneGrowing Season through practicaldemonstrations of available technologiesand public education in all media

Independent solaractivist in the Bostonarea

in developing an interface to enable on-line debates, Janusz Parfienuikand TopCoder, Inc. for their development work, and our experts, mod-erators, advisors and other community members for volunteering theirtime to this project.

References

1. Introne J, Laubacher R, Olson G, Malone T (2011) The climateCoLab: large scale model-based collaborative planning. In: 2011international conference on collaboration technologies and sys-tems (CTS), pp 40–47

2. Churchman CW (1967) Guest editorial: wicked problems. ManagSci 14(4):B141–B142

3. Rittel HWJ, Webber MM (1973) Dilemmas in a general theory ofplanning. Policy Sci 4(2):155–169

4. Conklin J (2005) Dialogue mapping: building shared understand-ing of wicked problems. Wiley, New York

5. Levin K, Cashore B, Bernstein S, Auld G, Student D (2007)Playing it forward: path dependency, progressive incrementalism,and the ‘super wicked’ problem of global climate change. In: In-ternational studies association 48th annual convention, February,vol 28

6. Nunamaker JF, Dennis AR, Valacich JS, Vogel D, George JF(1991) Electronic meeting systems. Commun ACM 34(7):40–61

7. DeSanctis G, Gallupe RB (1987) A foundation for the study ofgroup decision support systems. Manag Sci 33(5):589–609

8. Malone TW, Laubacher R, Dellarocas C (2010) The collective in-telligence genome. MIT Sloan Manag Rev 51(3):21–31

9. Climate CoLab credits, 03-Aug-2010. Online available: http://www.climatecolab.org/web/guest/resources/-/wiki/Main/Credits.Accessed: 04-Aug-2010 N1

10. Kulkarni AP, Can M, Hartmann B (2011) Turkomatic: automaticrecursive task and workflow design for mechanical turk. In: Pro-ceedings of the 2011 annual conference extended abstracts on hu-man factors in computing systems. New York, NY, USA, pp 2053–2058

11. Kittur A, Smus B, Khamkar S, Kraut RE (2011) CrowdForge:crowdsourcing complex work. In: Proceedings of the 24th annualACM symposium on user interface software and technology. NewYork, NY, USA, pp 43–52

12. Zhang H, Law E, Miller R, Gajos K, Parkes D, Horvitz E (2012)Human computation tasks with global constraints. In: Proceedingsof the 2012 ACM annual conference on human factors in comput-ing systems. New York, NY, USA, pp 217–226

52 Künstl Intell (2013) 27:45–52

13. Metz B, Davidson OR, Bosch PR, Dave R, Meyer LA Eds (2007)Contribution of working group III to the fourth assessment reportof the intergovernmental panel on climate change, 2007. Cam-bridge University Press, Cambridge

14. Parry ML, Canziani OF, Palutikof JP, van der Linden PJ, Eds CEHanson (2007) Contribution of working group II to the fourth as-sessment report of the intergovernmental panel on climate change,2007. Cambridge University Press, Cambridge, UK

15. Laubacher R (2012) What, where, who taxonomy—full view.The Climate CoLab, 2012. Online available: http://climatecolab.org/web/guest/resources/-/wiki/Main/What%2C+Where%2C+Who+taxonomy–Full+view. Accessed: 18-Sep-2012

16. Matthies M, Giupponi C, Ostendorf B (2007) Environmental de-cision support systems: current issues, methods and tools. EnvironModel Softw 22(2):123–127

17. Mayer IS, Van Bueren E, Bots P, van der Voort H, Seijdel R(2005) Collaborative decisionmaking for sustainable urban re-

newal projects: a simulation-gaming approach. Environ Plan B,Plan Des 32(3):403–423

18. Introne J, Laubacher R, Malone T (2011) Enabling open devel-opment methodologies in climate change assessment modeling.IEEE Softw 28(6):56–61

19. Climate CoLab model help. 10-Aug-2010. Online available:http://www.climatecolab.org/web/guest/resources/-/wiki/Main/Models+Help. Accessed: 04-Aug-2010N2

20. Salganik MJ, Dodds PS, Watts DJ (2006) Experimental study ofinequality and unpredictability in an artificial cultural market. Sci-ence 311(5762):854–856

21. Toplak J (2010) Preferential voting: definition and classification.In: Annual meeting of the Midwest political science association67th annual national conference. Chicago, IL

22. Smith WD (2010) Range voting. Dec-200023. Surowiecki J (2005) The wisdom of crowds. Anchor, New York