0205 f01 international research roadmap

0205F01_INTERNATIONAL RESEARCH ROADMAP

ICT Seventh Framework Programme (ICT FP7)

Grant Agreement No: 288828

Bridging Communities for Next Generation Policy-‐Making

Towards Policy-‐making 2.0: The International Research Roadmap on

ICT for Governance and Policy Modelling

Internal Deliverable Form

Project Reference No. ICT FP7 288828

Deliverable No. D2.2.2

Relevant Workpackage: WP2

Nature: Report

Dissemination Level: Public

Document version: FINAL 1.0

Date: 12/09/2013

Authors: David Osimo & Francesco Mureddu (T4I2), Riccardo Onori & Stefano Armenia (CATTID), Gianluca Carlo Misuraca (IPTS)

Reviewers: Eva Jaho (ATC), Andrea Bassi (MI)

Document description: This deliverable describes the final version of the new International Research Roadmap on ICT Tools for Governance and Policy


2 | P a g e

Modelling

History

Version Date Reason Revised by

1.0 30/06/2013 1st draft T4I2

2.0 12/07/2013 2nd draft sent for peer review

T4I2

26/07/2013 Peer review and feedback

ATC, MI

3.0 09/08/2013 3rd draft sent for final confirmation

T4I2

06/09/2013 Partners’ approval ATC, DIAG, W3C, IPTS, MI

1.0 12/09/2013 Final version sent to the PO and reviewers

ATC


3 | P a g e

TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................................................... 5 1. BACKGROUND: WHY A ROADMAP?........................................................................................................ 8

1.1. The rationale of the roadmap: what is the problem? ............................................................................. 8 1.2. An open and recursive methodology ...................................................................................................... 9 1.3. Scope and definition.............................................................................................................................. 16 1.4. Policy: Between politics and services .................................................................................................... 19

2. NOT JUST ANOTHER HYPE: THE DEMAND SIDE OF POLICY-‐MAKING 2.0 ................................................ 20 2.1. The typical tasks of policy-‐makers: the policy cycle .............................................................................. 21 2.2. The traditional tools of policy-‐making................................................................................................... 22 2.3. The key challenges of policy-‐makers ..................................................................................................... 23

2.3.1. Detect and understand problems before they become unsolvable............................................... 24 2.3.2. Generate high involvement of citizens in policy-‐making................................................................ 24 2.3.3. Identify “good ideas” and innovative solutions to long-‐standing problems .................................. 24 2.3.4. Reduce uncertainty on the possible impacts of policies ................................................................ 25 2.3.5. Ensure long -‐ term thinking ............................................................................................................ 27 2.3.6. Encourage behavioural change and uptake ................................................................................... 27 2.3.7. Manage crisis and the “unknown unknown” ................................................................................. 27 2.3.8. Moving from conversations to action ............................................................................................ 28 2.3.9. Detect non-‐compliance and mis-‐spending through better transparency ...................................... 28 2.3.10. Understand the impact of policies ............................................................................................... 29

2.4. When policy-‐making 2.0 becomes a reality: a tentative vision for 2030............................................... 29 2.4.1. Agenda setting phase: recognizing the problem ............................................................................ 29 2.4.2. Policy design ................................................................................................................................... 30 2.4.3. Implementation.............................................................................................................................. 31 2.4.4. Evaluation....................................................................................................................................... 31

2.5. The key challenges for policy makers and the corresponding phases in the policy cycle ..................... 32 3. THE SUPPLY SIDE: CURRENT STATUS AND THE RESEARCH CHALLENGES................................................ 33

3.1. Policy Modelling .................................................................................................................................... 33 3.1.1. Systems of Atomized Models ......................................................................................................... 33 3.1.2. Collaborative Modelling ................................................................................................................. 42 3.1.3. Easy Access to Information and Knowledge Creation .................................................................... 53 3.1.4. Model Validation ............................................................................................................................ 56 3.1.5. Immersive Simulation..................................................................................................................... 59 3.1.6. Output Analysis and Knowledge Synthesis..................................................................................... 61

3.2. Data-‐powered Collaborative Governance ............................................................................................. 64 3.2.1. Big Data .......................................................................................................................................... 64 3.2.2. Opinion Mining and Sentiment Analysis......................................................................................... 78 3.2.3. Visual Analytics for collaborative governance: the opportunities and the research challenges.... 85 3.2.4. Serious Gaming for Behavioural Change ........................................................................................ 98 3.2.5. Linked Open Government Data .................................................................................................... 103 3.2.6. Collaborative Governance ............................................................................................................ 109 3.2.7. Participatory Sensing .................................................................................................................... 113 3.2.8. Identity Management................................................................................................................... 117 3.2.9. Global Systems Science ................................................................................................................ 120

4. THE CASE FOR POLICY-‐MAKING 2.0: EVALUATING THE IMPACT .......................................................... 127 4.1. Cross analysis of case studies .............................................................................................................. 127

4.1.1. Global Epidemic and Mobility Model ........................................................................................... 128 Impact of Gleam......................................................................................................................................... 128 4.1.2. UrbanSim...................................................................................................................................... 129


4 | P a g e

4.1.3. Opinion Space............................................................................................................................... 130 4.1.4. 2050 Pathways Analysis................................................................................................................ 132 4.1.5. Cross analysis of the case studies................................................................................................. 134

4.2. Survey of Users’ needs results............................................................................................................. 136 4.3. Analysis of the prize winners............................................................................................................... 139 4.4. Lessons learnt from cases and prize.................................................................................................... 143 4.5. An additional research challenge: counterfactual impact evaluation of Policy Making 2.0................ 144

5. CONCLUSIONS: POLICY-‐MAKING 2.0 BETWEEN HYPE AND REALITY .................................................... 149 6. REFERENCES ....................................................................................................................................... 153 7. LIST OF ACRONYMS ............................................................................................................................ 157

LIST OF FIGURES Figure 1: the fragmentation of policy-‐making 2.0.................................................................................................. 8 Figure 2 Outline of the participatory process ...................................................................................................... 10 Figure 3: Policy Cycle and Related Activities ........................................................................................................ 22 Figure 4: Total Disasters Reported ...................................................................................................................... 28 Figure 5: Agricultural Production and Externalities Simulator (APES) ............................................................... 36 Figure 6: Conversational Modelling Interface .................................................................................................... 45 Figure 7: the PADGET Framework ....................................................................................................................... 46 Figure 8: the Time-‐Space Matrix ......................................................................................................................... 49 Figure 9: COMA, COllaborative Modelling Architecture .................................................................................... 50 Figure 10: OCOPOMO eParticipation Platform................................................................................................... 51 Figure 11: Twitrratr.............................................................................................................................................. 81 Figure 12: Wordclouds......................................................................................................................................... 82 Figure 13: UserVoice............................................................................................................................................ 82 Figure 14 Open Data Business Model (source: Istituto Superiore Mario Boella) .............................................. 106 Figure 15 -‐LOD providers and their linkages ...................................................................................................... 107 Figure 16 Rating other opinions' in Opinion Space ............................................................................................ 131 Figure 17 Playing the My2050 game for the demand side................................................................................. 133 Figure 18 Adoption of ICT Tools and Methodologies for policy-‐making (source: CROSSOVER Survey of Users’ Needs 2012) ....................................................................................................................................................... 137 Figure 19 Needs and Challenges in the Policy Making Process (source: CROSSOVER Survey of Users’ Needs 2012) .................................................................................................................................................................. 138 Figure 20: a proposed evaluation framework for policy-‐making 2.0 ................................................................. 144 Figure 21: Relation Between Policy-‐Making Needs and Research Challenges................................................... 149


5 | P a g e

Executive Summary This deliverable introduces and describes the interim version of the new International Research Roadmap on ICT tools for Governance and Policy Modelling, renamed by the project team as “Policy-‐Making 2.0”, one of the core outputs of the Crossover project, which is developed under WP2 Content Production.

The roadmap aims to establish the scientific and political basis for long-‐lasting interest and commitment to next generation policy-‐making by researchers and policy-‐makers. In doing so, it contains an analysis of what technologies are currently available, for what concrete purposes, and what could become available in the future. The main rationale for such a document is the current fragmentation of the landscape between different stakeholders, disciplines, policy domains and geographical areas.

The document is the result of a highly participative process undergone between the first draft and the final roadmap, with the involvement of hundreds of people through 11 different input methods, from live workshops to online discussion.


6 | P a g e

After a brief introduction of the background, the document analyses the demand side: the current status of policy-‐making, with the key tasks (illustrated by the traditional policy cycle) and existing challenges:

a. Detect and understand problems before they become unsolvable

b. Generate high involvement of citizens in policy-‐making

c. Identify “good ideas” and innovative solutions to long-‐standing problems

d. Reduce uncertainty on the possible impacts of policies

e. Ensure long -‐ term thinking

f. Encourage behavioural change and uptake

g. Manage crisis and the “unknown unknown”

h. Moving from conversations to action

i. Detect non-‐compliance and mis-‐spending through better transparency j. Understand the impact of policies

It then presents a concrete tentative vision of how policy-‐making could look in 2030, if these challenges were overcome.

Section 3 represents the core of the roadmap and presents the key research challenges to be addressed to achieve this vision, updating the original version based on the input of the consultation. For each research challenge, it presents the current status, the existing gaps, and short and long term research perspectives. The key research challenges are: 1. Policy Modelling

1.1. Systems of Atomized Models

1.2. Collaborative Modelling

1.3. Easy Access to Information and Knowledge Creation

1.4. Model Validation

1.5. Immersive Simulation

1.6. Output Analysis and Knowledge Synthesis

2. Data-‐powered Collaborative Governance

2.1. Big Data

2.2. Opinion Mining and Sentiment Analysis

2.3. Visual Analytics for collaborative governance: the opportunities and the research challenges

2.4. Serious Gaming for Behavioural Change

2.5. Linked Open Government Data

2.6. Collaborative Governance

2.7. Participatory Sensing

2.8. Identity Management

2.9. Global Systems Science

But to what extent policy-‐making 2.0 can be said to genuinely improve policy-‐making? Section 4 looks at the available evidence about the impact of policy-‐making 2.0, across case studies, the survey and the prize. As it emerges that no robust impact evaluation is available, we propose an additional


7 | P a g e

research challenge on impact evaluation of policy-‐making accompanied by a proposed evaluation framework.

Finally, we summarize the findings of the document bringing together the different sections, suggesting that policy-‐making 2.0 cannot be considered the panacea for all issues related to bad public policies, but that at the same time it is more than just a neutral set of disparate tools. It provides an integrated and mutually reinforcing set of methods that share a similar vision of policy-‐making and that should be addressed in an integrated and strategic way; and it provides opportunities to improve the checks and balances systems behind decision making in government, and as such it should be further pursued.

and as such it should be further pursued.


8 | P a g e

1. BACKGROUND: WHY A ROADMAP?

1.1. The rationale of the roadmap: what is the problem? The CROSSOVER project aims to consolidate and expand the existing community on ICT for Governance and Policy Modelling (built largely within FP7) by: -‐ Bringing together and reinforcing the links between the different global communities of researchers and experts: it will create directories of experts and solutions, and animate knowledge exchange across communities of practice both offline and online; -‐ Reaching out and raising the awareness of non-‐experts and potential users, with special regard to high-‐level policy-‐makers and policy advisors: it will produce multimedia content, a practical handbook and high-‐level policy conferences with competition for prizes; -‐ Establishing the scientific and political basis for long-‐lasting interest and commitment to next generation policy-‐making, beyond the mere availability of FP7 funding: it will focus on use cases and a demand-‐driven approach, involving policy-‐makers and advisors. The CROSSOVER project pursues this goal through a combination of content production, ad hoc and well-‐designed online and offline animation; as well as strong links with existing communities outside the CROSSOVER project and outside the realm of e-‐Government. The present deliverable is one of the core outputs of the project: the International Research Roadmap on ICT Tools for Governance and Policy Modelling. It aims to create a common platform between actors fragmented in different disciplines, policy domains, organisations and geographical areas, as illustrated in the figure below.

Figure 1: the fragmentation of policy-‐making 2.0 But most of all, it aims to provide a clear outline of what technologies are available now for policy-‐makers to improve their work, and what could become available tomorrow.


9 | P a g e

CROSSOVER builds on the results of the CROSSROAD project1, which elaborated a research roadmap on the same topic along the whole of 2010. With respect to the previous roadmap, this document is firstly a revised and updated version. Beside this, it contains some fundamental novelties:

-‐ A demand-‐driven approach: rather than focussing on the technology, the present roadmap starts from the needs and the activities of policy-‐making and then links the research challenges to them.

-‐ An additional emphasis on cases and applications: for each research challenge, we indicate relevant cases and practical solutions

-‐ A clearer thematic focus on ICT for Governance and Policy-‐Modelling, by dropping more peripheral grand challenges of Government Service Utility and Scientific Base for ICT-‐enabled Governance

-‐ A global coverage: while CROSSROAD focussed on Europe, CROSSOVER includes cases and experiences from all over the world

-‐ A living roadmap: the present deliverable is accompanied by an online repositories of tools, people and applications

1.2. An open and recursive methodology

The present Research Roadmap on Policy-‐Making 2.0 is developed with a sequential approach based on the existing research roadmap developed by the CROSSROAD project. In order to achieve the goals of overcoming the fragmentation, an open and inclusive approach was necessary.

In the initial phase of the project, up to M6 (March 2012), the consortium started a collection of literature, information about software tools and applications cases. In addition to this desk-‐based review, the document has benefited from the informal discussions being held on the LinkedIn group of the project (Policy-‐making 2.0), where more than 800 practitioners and researchers are discussing the practices and the challenges of policy-‐making.

The first draft of the roadmap was then released in M9 (June 2012) of the project, for public feedback. The publication of the deliverable kicked off the engagement activities of the project, designed to provide further input and to improve the roadmap:

-‐ As soon as it was released, the preliminary version of the roadmap was published in

commentable format on the project website http://www.CROSSOVER-‐project.eu/.

Animators stimulated discussion about it and generated comments by researchers and

practitioners alike. This participatory process helped enriching the roadmap, which was then

published in its final version after validation by the community/ies of practitioners and policy

makers

-‐ Two workshops organised by the project aimed at gathering input on the research

challenges and feedback on the proposed roadmap

-‐ An online survey, as well as several focus groups and meetings with practitioners from civil

society and government helped to focus the roadmap on the actual needs

1 http://CROSSROAD.epu.ntua.gr/


10 | P a g e

Figure 2 Outline of the participatory process

The process for updating the roadmap included therefore a wide set of contributions. Firstly, the Crossroad roadmap was enriched with desk-‐based research: 202 cases collected in the platform + 4 cases collected and described in the case studies performed by the National Technical University of Athens (NTUA), and the 50 applications to the prize.

This first draft was then published for comments by some of the 800 members of the LinkedIn group who also provided relevant cases. An additional survey of users’ needs provides provided insights from 240 respondents and over 200 people presents presented at focus groups. Additional discussions with Global Systems Science community, third party workshops and the US Policy Informatics Network helped in refine refining further the roadmap.

The two workshops provided high-‐quality insight that enriched the roadmap with specific contributions.

In the table below we outline in detail the specific contribution of each section of the roadmap, that is described in full in the following section.


11 | P a g e

Type of contribution Extent of the contribution Contribution to the roadmap

1) Comments to the roadmap • 40 comments • 9 different experts

• Visual Analytics • Systems of Atomized Models • Model Validation • Serious Gaming

2) Presentations in the PMOD

workshop

• Papers received: 42 • Registered participants: 70 • No. Countries’ citizens present:

20

• Linked Open Government Data

3) Presentations in the

Transatlantic workshop

• 16 presentations • 30 participants

• Collaborative Modelling • Systems of Atomized Models • Opinion Mining

4) Survey of User’s Needs

• 236 respondents • 33% engaged in policy design • 27% engaged in monitoring and evaluation

• 22% engaged in agenda setting • 18% engaged in policy implementation

• Impact of policy making 2.0 • Roadmap methodology • Linked Open Government Data • Opinion Mining • Collaborative Governance

5) Focus groups

139 attendants -‐ Forum PA, the Italian leading conference on e-‐government

• 35 attendants-‐ INSITE event on sustainability

• 40 attendants -‐ Webinar for the United Nations Development Programme

• Impact of policy making 2.0 • Roadmap methodology

6) Case studies • Collection of 202 tools and practices

• Elicitation of 20 best practices • Further elicitation of 4 best

practices for in-‐depth case study

• Impact of policy making 2.0 • Roadmap methodology • Annex with a repository of cases

7) Analysis of the prize • 47 submission received • 10 short listed • 3 winners

• Analysis of the prize process on the Impact Chapter

8) LinkedIn group • 840 participants • Comments to the roadmap • Increased attendance to the

workshops • Collection of practices and tools

Table 1 Contributions to the roadmap

1) Comments to the Roadmap

The roadmap has been published in commentable format in two different versions: a short one on Makingspeechtalk2, and a full version (downloadable after answering the survey on the needs of

2 http://makingspeechestalk.com/CROSSOVER/


12 | P a g e

policy-‐makers) available in the CROSSOVER website3. Everybody was able to comment on single parts of the roadmap or to propose new topics, application cases and research challenges. The aim of publishing the document in commentable format was to get the input from experts for co-‐creating the roadmap. More specifically we were interested in knowing if the current formulation of the research challenge was acceptable, and we wanted to collect best practices and application cases from the community of experts and practitioners at large. As already mentioned, the roadmap received over 40 useful and detailed comments from a number of experts in the different domains.

2) PMOD Workshop

The June 2012 workshop was the first of three to be organised under the CROSSOVER project. Formally titled "Using Open Data: policy modelling, citizen empowerment, data journalism" but generally referred to by the term PMOD (policy modelling), it set out to explore whether advocates' claims of the huge potential for open data as an engine for a new economy, as an aid to transparency and, of particular relevance to CROSSOVER, as an aid to evidence-‐based policy modelling, were justified. In terms of organization, the event was run as a W3C/CROSSOVER workshop and held at the European Commission's Albert Borschette Conference Centre in the two days immediately prior to the Digital Agenda Assembly. That combination helped to secure good support from a high calibre audience. 42 papers were received and the majority was accepted by the programme committee for full presentation. Authors of several other papers plus members of the programme committee, the CROSSOVER animators and a small number of invited guests comprised the 70 registered attendees of which 67 turned up. The event reached a larger audience through organising a networking event on the evening following the workshop to which attendees of the data workshop at the Digital Agenda Assembly were invited. Furthermore, through the live IRC channel and Tweets using the #pmod hashtag, others were able to monitor proceedings. The agenda, attendee list and final report are all available on the W3C Web site which provides a high profile for the workshop and the project.

Most of the results of the workshop were used to improve the research challenge on Linked Open Government Data.

3) Transatlantic Workshop

The Transatlantic Research on Policy Modelling Workshop that was held in Washington, DC on January 28th and 29th, 2013. It was organized by the Millennium Institute and the New America Foundation (NAF), Washington, DC, USA. NAF is a nonprofit, nonpartisan public policy institute that invests in new thinkers and new ideas to address the next generation of challenges facing the United States. This event brought together speakers and attendees working and/or interested in improving ICT tools for education and policy makers. The speakers and attendees came from a diverse background, both technical and non-‐technical to share experiences and knowledge and discuss ways to make the current state of modelling and ICT more accessible and attractive for decision makers on both sides of the Atlantic Ocean. The models presented in the workshop have been integrated in the “Collaborative Modelling”, “Systems of Atomized Models” and “Opinion Mining” research challenges.

4) Survey of User’s Needs

3 http://www.CROSSOVER-‐project.eu/ResearchRoadmap.aspx


13 | P a g e

The Survey of Users’ Needs performed within the scope of the CROSSOVER project aimed at collecting the views and the requirements of policy-‐making stakeholders. More in particular the survey intended to stimulate actual and potential practitioners, such as decision makers (government official involved in the policy-‐making process) or policy advisors (technical expert advising decision-‐makers from outside government) to provide input, feedback and validation to the new research roadmap on ICT tools for Governance and Policy Modelling under development (CROSSOVER, 2012b). About 450 people took part in the overall exercise, combining live meetings (214) and online survey (240+ answers), providing concrete elements to improve the CROSSOVER roadmap and the other activities to be carried out by the project.

5) Focus groups

In addition to the survey, Tech4i2 ran a series of dedicated meetings where the roadmap was presented and followed up by intense dedicated discussion. These events where all high-‐profile, attended by policy-‐makers in the broad sense: not only government officials, but also policy advisors and civil society organisations. More precisely three events have been run:

• On the 17th of May 2012 CROSSOVER was invited to give a keynote speech to ForumPA on the CROSSOVER Research Roadmap. FORUM PA is a leading European exhibition exploring innovation in Public Administration and local systems. For 22 years, FORUM PA has attracted thousands of visitors and hundreds of exhibitors (public authorities, private companies and citizens) to come together and learn and the participation of important leaders: ministers, Nobel prize winners (Amartya Sen, Edward Prescott), industry leaders (Luca Cordero di Montezemolo) and hundreds of speakers.

• On May 24th 2012, CROSSOVER was invited to attend the HUB/Insite project meeting of sustainability practitioners from all over Europe. The Hub and the INSITE Project brought together more than 25 sustainability practitioners working at the cutting edge of innovation within industry, urban development, energy, technology and policy across Europe. This includes people tackling today’s key challenges in carbon reduction, smart cities, governance and behavioural change across all these areas. Tech4i2 presented the Research Roadmap, and facilitated a dedicated session CROSSOVER was invited to attend the HUB/Insite project meeting of sustainability practitioners from all over Europe.

• On March 22nd 2012, CROSSOVER was invited to present the policy-‐making 2.0 model to the practitioners of the “governance” network of UNDP – Europe and CIS, which included about 40 people from Central and Eastern Europe. Webinar for the United Nations Development Programme – Europe and CIS

6) Case Studies

Within the scope of the CROSSOVER project, the European Commission's Joint Research Centre, Institute for Prospective Technological Studies (JRC-‐IPTS), in collaboration with a team of experts of the National Technical University of Athens (NTUA) carried out the activity of mapping and identification of Case Studies on ICT solutions for governance and policy modelling (CROSSOVER, 2013). The research design envisaged a set of macro phases. The initial phase consisted in the creation of a case study repository through the identification and prioritization of potential sources of information, an open invitation for proposal of cases through web2.0 channels, followed by the definition of the 1st-‐round criteria for selecting at least twenty practices and the information-‐oriented selection of the corresponding case studies on applications of ICT solutions for governance and policy modelling. In the second phase, case studies have been elicited through the definition of the 2nd-‐round criteria for selecting eight promising practices and the application of a multi-‐criteria method, followed by further elaboration on the eight case studies that have been selected by the


14 | P a g e

multi-‐criteria method based on desk research. In the third phase the final four cases have been selected and subjected to an in-‐depth analysis carried out through meticulous study of the available public documentation and the conduction of interviews with key involved stakeholders. After the final selection of cases and the in depth analysis, the findings have been synthesized through the analysis of the emerging trends from applications of ICT solutions for governance and policy modelling as well as the development of key considerations for the CROSSOVER roadmap for the themes that refer to its scope. Finally the key findings of the analysis of the four cases have been shared with the CROSSOVER partners and the community that follows closely the Policy Making 2.0 domain over various Web 2.0 channels, to provide feedback and validation. The key results of the case studies are described later in the impact section.

7) Analysis of the Prize

This prize was given to the best policy-‐making 2.0 applications, that is are for the best use of technology to improve the design, delivery and evaluation of Government policy. The focus of the jury has been on implementations that can show a real impact on policy making, either in terms of better policy or wider participation. These technologies included, but are not limited to:

• Visual analytics

• Open and big data

• Modelling and simulation (beyond general equilibrium models)

• Collaborative governance and crowdsourcing

• Serious gaming

• Opinion mining

An important condition for participating to the selection has been the real-‐life implementation of technology to policy issues.

Out of 50 applications, the jury selected the best 12 and eventually the 3 winners, which received an IPAD mini. The principal domains of the applications were as follow:

• 23 in the “Collaborative Governance and Crowd-‐sourcing” domain • 13 in the “Open and Big Data” domain • 4 in the “Visual Analytics” domain • 2 in the “Modelling and Simulation (beyond general equilibrium models)” domain • 2 in the “Serious Gaming” domain • 1 in each of the following domains: “Open Source Governance”, “Opinion Mining”,

“Participatory Policy Making”

All the relevant applications received have been integrated in the roadmap. The criteria for judging the applications were:

• Impact on the quality of policies • Openness, scalability and replicability • Extensiveness of public and policymakers’ take up • Technological innovativeness

To this respect, the applicants to the prize were required to provide the following information:

• Name of the application


15 | P a g e

• Year of launch • Short description of the technological domain • Link to the application • Describe the impact of the application on the quality of policies • Describe the public and policymaker take up of the application • Describe to what extent the application was technologically innovative • Contact details of the applicant

8) LinkedIn Group Policy-‐Making 2.0

A crucial element in the engagement of stakeholders is given by the creation of a group on LinkedIn called Policy Making 2.04, which is a virtual place where actual and potential practitioners of advanced ICT tools for policy-‐making can exchange experiences. The group displays a high selected pool of high level members (over 840) engaging in discussions and exchange of views. In order to foster debate in the group, the CROSSOVER consortium posts on a regular base info about the new cases and tools to be integrated in the knowledge repository. Some other discussion topics relate to the best ways to engage the government in online policy making, the posting of third parties content and info about incoming CROSSOVER workshops. In particular the group is being used for disseminating the Survey on the ICT Needs of Policy Makers, as well as the roadmap in commentable format. The Policy Making 2.0 group also serves as a liaison channel with similar projects such as eGvoPoliNet and OCOPOMO. As agreed the eGovPoliNet LinkedIn group has merged with the CROSSOVER Policy Making 2.0 group, and after the end of the CROSSOVER project the interaction will continue led by the eGovPoliNet consortium. Moreover as we are approaching the end of the project we decided to shift from a closed LinkedIn group to an open one.

4 http://www.linkedin.com/groups?home=&gid=4165795


16 | P a g e

1.3. Scope and definition

Policy-‐making 2.0 refers to a set of methodologies and technological solutions aimed at innovating policy-‐making. As we will describe in section 2.1, the scope goes well beyond the focus on “Decision-‐making” notion typical of eParticipation, and encompasses all phases of the policy cycle. The main goal is limited to improving the quality of policies, not of making them more consensual or representative.

Policy-‐making 2.0 is a new term that we have coined to express in more understandable terms the somehow technical notion of “ICT for governance and policy modelling”. Its usage in the course of the project proved more effective than the latter when discussing with stakeholders. Thereby from now on we will refer to the roadmap as the Research Roadmap on Policy-‐Making 2.0.

The full set of methodologies and tools has been spelled out in the taxonomy in WP15: 1.1. Open government information & intelligence for transparency

1.1.1. Open & Transparent Information Management 1.1.1.1. Open data policy 1.1.1.2. Open data licence 1.1.1.3. Open data portal 1.1.1.4. Code list 1.1.1.5. Vocabulary/ontology 1.1.1.6. Reference data 1.1.1.7. Data cleaning and reconciliation tool

1.1.2. Data published on the Web under an open licence 1.1.2.1. Human-‐readable data 1.1.2.2. Machine readable data in proprietary format 1.1.2.3. Machine-‐readable data published in a non-‐proprietary format 1.1.2.4. Data published in RDF 1.1.2.5. SPARQL endpoint for querying RDF data 1.1.2.6. RDF data linked to other data sets

1.1.3. Visual Analytics 1.1.3.1. Visualisation of a single, static, embedded data set 1.1.3.2. Visualisation of multiple static data sets 1.1.3.3. Visualisation of a single live data feed or updating data set 1.1.3.4. Visualisation of multiple data points, including live feeds or updates

1.2. Social computing, citizen engagement and inclusion 1.2.1. Social Computing

1.2.1.1. Collaborative writing and annotation 1.2.1.2. Content syndication 1.2.1.3. Feedback and reputation management systems 1.2.1.4. Social Network Analysis 1.2.1.5. Participatory sensing

1.2.2. Citizen Engagement

5 The taxonomy presented here builds on CROSSROAD taxonomy, which has been expanded, reviewed and updated by the members of the Consortium


17 | P a g e

1.2.2.1. Online deliberation 1.2.2.2. Argumentation support 1.2.2.3. Petition, Polling and voting 1.2.2.4. Serious games 1.2.2.5. Opinion mining

1.2.3. Public Opinion-‐Mining & Sentiment Analysis 1.2.3.1. Opinion tracking 1.2.3.2. Multi-‐lingual and Multi-‐Cultural opinion extraction and filtering 1.2.3.3. Real-‐time opinion visualisation 1.2.3.4. Collective Wisdom Analysis and Exploitation

1.3. Policy Assessment 1.3.1. Policy Context Analysis

1.3.1.1. Forecasting 1.3.1.2. Foresight 1.3.1.3. Back-‐Casting 1.3.1.4. Now-‐Casting 1.3.1.5. Early Warning Systems 1.3.1.6. Technology Road-‐Mapping (TRM)

1.3.2. Policy Modelling 1.3.2.1. Group Model Building 1.3.2.2. Systems Thinking & Behavioural Modelling 1.3.2.3. System Dynamics 1.3.2.4. Agent-‐Based Modelling 1.3.2.5. Stochastic Modelling 1.3.2.6. Cellular Automata

1.3.3. Policy Simulation 1.3.3.1. Multi-‐level & micro-‐simulation models 1.3.3.2. Discrete Event Simulation 1.3.3.3. Autonomous Agents, ABM Simulation, Multi-‐Agent Systems (MAS) 1.3.3.4. Virtual Worlds, Virtual Reality & Gaming Simulation 1.3.3.5. Model Integration 1.3.3.6. Model Calibration & Validation

1.3.4. Policy Evaluation 1.3.4.1. Impact Assessment 1.3.4.2. Scenarios 1.3.4.3. Model Quality Evaluation 1.3.4.4. Multi-‐Criteria Decision Analysis

1.4. Identity, privacy and trust in governance 1.4.1. Identity Management

1.4.1.1. Federated Identity Management Systems 1.4.1.2. User centric, self managed and lightweight credentials 1.4.1.3. Legal-‐social aspects of eIdentity management 1.4.1.4. Mobile Identity (Portability)

1.4.2. Privacy 1.4.2.1. Privacy and Data Protection 1.4.2.2. Privacy Enhancing Technologies 1.4.2.3. Anonymity and Pseudonymity 1.4.2.4. Open data management (including Citizen Profiling, 'digital shadow' tracing and tracking

1.4.3. Trust


18 | P a g e

1.4.3.1. Legal Informatics 1.4.3.2. Digital Rights Management 1.4.3.3. Digital Citizenship Rights and feedback loops 1.4.3.4. Intellectual Property in the digital era 1.4.3.5. Trust-‐building Services (including data processing and profiling by private actors for public services)

1.5. Future internet for collaborative governance 1.5.1. Cloud Computing

1.5.1.1. Cloud service level requirements 1.5.1.2. Business models in the cloud 1.5.1.3. Cloud interoperability 1.5.1.4. Security and authentication in the cloud 1.5.1.5. Data confidentiality and auditability 1.5.1.6. Cloud legal implications

1.5.2. Pervasive Computing & Internet of Things in Public Services 1.5.2.1. Ambient intelligence 1.5.2.2. Exploiting smart objects 1.5.2.3. Standardization 1.5.2.4. Business models for pervasive technologies 1.5.2.5. Privacy implications and risks

1.5.3. Provision of next generation public e-‐services 1.5.3.1. Fixed and mobile network access technologies 1.5.3.2. Mobile web 1.5.3.3. Models for information dissemination 1.5.3.4. Management of scarce network capacity and congestion problems 1.5.3.5. Large-‐scale resource sharing 1.5.3.6. Interworking of different technologies for seamless connectivity of users

1.5.4. Future Human/Computer Interaction Applications & Systems 1.5.4.1. Web accessibility 1.5.4.2. User-‐centered design 1.5.4.3. Augmented cognition 1.5.4.4. Human senses recognition

Policy-‐making 2.0 encompasses clearly a wide set of methodologies and tools. At first sight, it might appear unclear what the common denominator is. In our view, what they share is that they are designed to use technology in order to inform the formulation of more effective public policies. In particular, these technologies share a common approach in taking into account and dealing with the full complexity of human nature. As spelled out originally in the CROSSOVER project proposal: “traditional policy-‐making tools are limited insofar they assume an abstract and unrealistic human being: rational (utility maximizing), consistent (not heterogeneous), atomised (not connected), wise (thinking long-‐term) and politically committed (as Lisa Simpson)”. Policy-‐making 2.0 thus accounts for this diversity. Its methodologies and tools are designed not to impose change and artificial structures, rather to interact with this diversity. Agent-‐based models account for the interaction between agents that are different in nature and values; systems thinking accounts for long-‐term interacting impacts; social network analysis deals with the mutual influences between people rather than fully rational choices; big data analyses observed behaviour rather than theoretical models; persuasive technologies deal with the complex psychology of individuals and introduces gaming values to involve more “casual” participants. Moreover, policy-‐making 2.0 tools allow all stakeholders to participate to the decision-‐making process.


19 | P a g e

1.4. Policy: Between politics and services

The application of technology to governmental issues is not a new topic. Indeed e-‐government and the new buzzword of government 2.0, have become mainstream in recent years: how and why a future looking research agenda could still refer to the 2.0 paradigm as innovative? The novelty lies in the “policy” part of the definition.

So far, the application of "2.0" technologies to governmental processes has focussed mainly on the usage of social media for political communication, best exemplified by the Obama campaign. The typical narrative is that in the age of social media, traditional communication campaigns and political parties are unsuited to generate commitment and action by citizens, which instead want to take active part in the campaign and self-‐organize via social media: ""A candidate who can master the Internet will not only level the playing field; he will level the opposition." RightClick Strategies' Larry Purpuro.

A second area of strong focus proved to be the collaborative provision of public services based on peer-‐to-‐peer support and open data, best exemplified by the widely spread "appsfordemocracy" contests. The narrative here is that government should act as a platform and enable third parties (and citizens themselves) to co-‐create and deliver public services based on open government data. This is what Goldsmith and Eggers (2004) call "governing by network".

Indeed, the Obama administration clearly shows these priorities, moving from state-‐of-‐the-‐art campaigning in order to be elected, and then implementing a strong open data policy with crowdsourcing initiatives to let citizens create services based on these data.

Between "politics" and "public services co-‐delivery", much less attention has been devoted to the usage of social technology to improve public policy. While politics deal with the legislative branch, the Parliament, policy-‐making is mainly the realm of the executive branch. Typically, the job of policy-‐making involves a great deal of socio-‐economic analysis as well as consultation with stakeholders.

This roadmap aims to fill this gap, by providing a complete picture of how technology can improve policy-‐making.


20 | P a g e

2. Not just another hype: the Demand side of policy-‐making 2.0 In the context of new technologies, we are periodically informed about the emerging wave that will change everything, only to see it quickly forgotten after years or even month in what Gartner calls “trough of disillusionment”. While some of this emphasis is certainly driven by commercial interests, in many other cases it reflects a genuine optimism of its proponents, who tend to underestimate the real-‐life bottlenecks to adoption by less enthusiast people.

Movzorov critically calls this cyber-‐utopianism or technological solutionism (Morozov 2013); on a similar note, many years of eGovernment policy have revealed the fundamental importance of non-‐technological factors, such as organisational change, skills, incentives and culture.

One way to prevent policy-‐making 2.0 to become yet another hype in the Gartner curve, is to precisely spell out the challenges that these new technologies help to address. Indeed, the importance of this demand-‐driven approach based on grand challenges is fully embraced by the new Horizon2020 research programme of the European Union. 6 Furthermore, a demand-‐driven approach helps us to frame the technological opportunities in a language understandable to policy-‐makers, thereby supporting the awareness-‐raising objective of the CROSSOVER project.

When analysing the demand side, our first consideration is that policy-‐making is more important and complex than ever. The role of government has substantially changed over the last twenty years. Governments have to re-‐design their role in areas where they were directly involved in service provision, such as utilities but also education and health. This is not simply a matter of privatisation, or of a linear trend towards smaller government. Indeed, even before the recent financial turmoil and nationalisation of parts of the financial system, government role in the European societies was not simply “diminishing”, but rather being transformed. At the same time, it is increasingly recognized that the emergence of new and complex problems requires government to increasingly collaborate with non-‐governmental actors in the understanding and in the addressing of these challenges7. As an OECD report states the following:

“Government has a larger role in the OECD countries than two decades ago. But the nature of public policy problems and the methods to deal with them are still undergoing deep change. Governments are moving away from the direct provision of services towards a greater role for private and non-‐profit entities and increased regulation of markets. Government regulatory reach is also extending in new socio-‐economic areas. This expansion of regulation reflects the increasing complexity of societies. At the same time, through technological advances, government’s ability to accumulate information in these areas has increased significantly. As government face more new and complex problems that cannot be dealt with easily by direct public service provision, more ambitious policies require more complex interventions and collaboration with non-‐governmental parties”

This is particularly challenging in our "complex" societies. “Complex” systems are those where “the behaviour of the system as a whole cannot be determined by partitioning it and understanding the behaviour of each of the parts separately, which is the classic strategy of the reductionist physical sciences”. The present challenges governments must face, as described by the OECD, are complex as they are characterised by many non-‐linear interactions between agents; they emerge from these interactions and are therefore difficult to predict. The financial crisis is probably the foremost example of a complex problem, which proved impossible to predict with traditional decision-‐making tools.

6 http://ec.europa.eu/research/horizon2020/index_en.cfm?pg=h2020 7 See Ostrom: http://www.nobelprize.org/nobel_prizes/economics/laureates/2009/ostrom-‐lecture.html


21 | P a g e

2.1. The typical tasks of policy-‐makers: the policy cycle

Policy-‐making is typically carried out through a set of activities described as "policy-‐cycle" (Howard 2005). In this document we propose a new way of implementing policies, by first assessing their impacts in a virtual environment.

While different versions of the cycle are proposed in literature, in this context we adopt a simple version articulated in 5 phases:

-‐ agenda setting encompasses the basic analysis on the nature and size of problems at stakes are addressed, including the causal relationships between the different factors

-‐ policy design includes the development of the possible solutions, the analysis of the potential impact of these solutions8, the development and revision of a policy proposal

-‐ adoption is the cut-‐off decision on the policy. This is the most delicate and sensitive area, where accountability and representativeness are needed. It is also the area most covered by existing research on e-‐democracy

-‐ implementation is often considered the most challenging phase, as it needs to translate the policy objectives in concrete activities, that have to deal with the complexity of the real world . It includes ensuring a broader understanding, the change of behaviour and the active collaboration of all stakeholders.

-‐ Monitoring and evaluation make use of implementation data to assess whether the policy is being implemented as planned, and is achieving the expected objectives.

The figure below (authors’ elaboration based on Howard 2005 and EC 2009) illustrates the main phases of the policy cycle (in the internal circle) and the typical concrete activities (external circle) that accompany this cycle. In particular, the identified activities are based on the Impact Assessment Guidelines of the European Commission (EC 2009).

8 A very important element in policy design and formulation is given by ex-‐ante evaluation. In this respect ICT tools for policy-‐making can play an important role, simulating alternative policy options and impacts before implementing a policy action


22 | P a g e

Figure 3: Policy Cycle and Related Activities

Traditionally, the focus about the impact of technology in policy-‐making has been on the adoption phase, analysing the implications of ICT for direct democracy. In the context of the CROSSOVER project, we adopt a broader conceptual framework that embraces all phases of policy-‐making.

2.2. The traditional tools of policy-‐making

Let us present now what are the methodologies and tools already traditionally adopted in policy-‐making. Typically, in the agenda-‐setting phase, statistics are analysed by government and experts contracted by government in order to understand the problems at stake and the underlying causes of the problems. Survey and consultations, including online ones, are frequently used to assess the stakeholders’ priorities, and typically analysed in-‐house. General-‐equilibrium models are used as an assessment framework.

Once the problems and its causes are defined, the policy design phase is typically articulated through an ex-‐ante impact assessment approach. A limited set of policy options are formulated in house with


23 | P a g e

the involvement of experts and stakeholders. For each option, models are simulated in order to forecast possible sectoral and cross-‐sectoral impacts. These simulations are typically carried out by general-‐equilibrium models if the time frame is focused on short and medium term economic impacts of policy implementation. Based on the simulated impact, the best option is submitted for adoption.

The adoption phase is typically carried out by the official authority, either legislative or executive (depending on the type of policy). In some cases, decision is left to citizens through direct democracy, through a referendum or tools such as participatory budgeting; or to stakeholders through self-‐regulation.

The implementation phase typically is carried out directly by government, using incentives and coercion. It benefits from technology mainly in terms of monitoring and surveillance, in order to manage incentives and coercion, for example through the database used for social security or taxes revenues.

The monitoring and evaluation phase is supported by mathematical simulation studies and analysis of government data, typically carried out in-‐house or by contractors. Moreover, as numbers aggregate the impacts of everything that happens, including policy, it is difficult to single out the impacts of one policy ex post. Final results are published in report format, and fed back to the agenda setting phase.

2.3. The key challenges of policy-‐makers

Needless to say, the current policy-‐making process is seldom based on objective evidence and not all views are necessarily represented. Dramatic crises seem to happen too often, and governments struggle to anticipate and deal with them, as the financial crisis has shown. Citizens feel a sense of mistrust towards government, as shown by the decrease in voters turnout in the elections.

In this section, we analyse and identify the specific challenges of policy-‐making. The goal is to clearly spell out "what is the problem" in the policy making process that policy-‐making 2.0 tools can help to solve.

The challenges have been identified on desk-‐based research of "government failure" in a variety of contexts, and are illustrated by real-‐life examples.

One first overarching challenge is the emergence of a distributed governance model. The traditional division of “market” and “state” no longer fits a reality where public decision and action is effectively carried out by a plurality of actors. Traditionally, the policy cycle is designed as a set of activities belonging to government, from the agenda setting to the delivery and evaluation. However in recent years it has been increasingly recognized that public governance involves a wide range of stakeholders, who are increasingly involved not only in agenda-‐setting but in designing the policies, adopting them (through the increasing role of self-‐regulation), implementing them (through collaboration, voluntary action, corporate social responsibility), and evaluating them (such as in the case of civil society as watchdog of government). As Elinor Ostrom stated in her lecture delivered when receiving the Nobel Prize in Economics9: “A core goal of public policy should be to facilitate the development of institutions that bring out the best in humans. We need to ask how diverse polycentric institutions help or hinder the innovativeness, learning, adapting, trustworthiness, levels of cooperation of participants, and the achievement of more effective, equitable, and sustainable outcomes at multiple scales”. This acknowledgement leads to important implications for the

9 http://www.nobelprize.org/nobel_prizes/economics/laureates/2009/ostrom-‐lecture.html


24 | P a g e

CROSSOVER roadmap: policy-‐making 2.0 tools are not just tools for government, but for all stakeholders to participate in the policy-‐making process10.

2.3.1. Detect and understand problems before they become unsolvable

The continuous struggle for evidence-‐based policy-‐making can have some important and potentially negative implications in terms of the capacity of prompt identification of problems. Policy-‐makers have to balance the need for prompt reaction with the need for justified action, by distinguishing signal from noise. Delayed actions are often ineffective; at the same time, short-‐term evidence can lead to opposite effects. In any case, government have scarce resources and need to prioritize interventions on the most important problems.

For instance the significant underestimation of the risks of the housing bubble in the late 2000s, and the systemic reaction that it would lead to, led to delayed reactions11.

Systemic changes do not happen gradually, but become visible only when it's too late to intervene or the cost of intervening is too high. For example, ICT is today recognized as a key driver of productivity and growth, but evidence to prove this became available at a distance of years from the initial investment. In fact the initial lack of correlation between ICT investment and productivity growth was mostly due to incorrect measurement of ICT capital prices and quality. Subsequent methodologies found that computer hardware played an increasing role as a source of economic growth (see inter al. Colecchia and Schreyer 2002, Jorgenson and Stiroh 2000, Oliner and Sichel 2000).

The problem is in this case is therefore twofold: to collect data more rapidly; and to analyse them with a wider variety of models that account for systemic, long term effects and that are able to detect and anticipate weak signals or unexpected wild cards.

2.3.2. Generate high involvement of citizens in policy-‐making

The involvement of citizens in policy-‐making remains too often associated with short-‐termism and populism.

It is difficult to engage citizens in policy discussions in the first place: public policy issues are not generally appealing and interesting as citizens fail to understand the relevance of the issues and to see "what's in it for me". The decline in voters’ turnout and the lack of trust in politicians reflects this. More importantly, there are innumerable cases where the "right" policies are not adopted because citizens "would not understand" or because it is not politically acceptable.

While the Internet has long promised an opportunity for widespread involvement, e-‐participation initiatives often struggle to generate participation. Participation is often limited to those that are already interested in politics, rather than involving those that are not.

When participation occurs, online debates tend to focus on eye-‐catching issues and polarized positions, in part because of the limits of the technology available. It is extremely difficult and time consuming to generate open, large scale and meaningful discussion.

2.3.3. Identify “good ideas” and innovative solutions to long-‐standing problems

10 However in our project we mainly focus on tools that are used or can be adopted by Governments, otherwise we would risk to enlarge too much the scope of the research roadmap 11 http://www.wsws.org/en/articles/2013/01/26/fede-‐j26.html


25 | P a g e

Innovation in policy-‐making is a slow process. Because of the technical nature of issues at hand, the policy discussion is often limited to restricted circles. Innovative policies tend to be "imported" through "institutional isomorphism". Innovative ideas, from both civil servants and citizens, fail to surface to the top hierarchy and are often blocked for institutional resistance.

Existing instruments for large-‐scale brainstorming remain limited in usage, and fail to surface the most innovative ideas. Crowdsourcing typically focus on the most “attractive” ideas, rather than the most insightful.

2.3.4. Reduce uncertainty on the possible impacts of policies

When policy options have been developed, simulations are carried out to anticipate the likely impact of policies. The option with the most positive impact is normally the one that is proposed for adoption.

Most existing methodologies and tools for the simulation of policy impacts work decently with well known, linear phenomena. However, they are not effective in times of crisis and fast change, which unfortunately turn out to be exactly the situations where government intervention is most needed.

As an example nowadays the European Central Bank bases its analysis of the EURO Area economy and monetary policy on a derived version of the DSGE model developed by Frank Smet and Raf Wouters in 200312. Smet and Wouters’ model is deeply microfounded, allowing for a rigorous theoretical structure of the model. Moreover in this setting the reduced form parameters are related to deep structural parameters in order to mitigate Lucas’ critique, while the utility of agents can be taken as a measure of welfare in the economics (Phelps ed. 1970).

However, the DSGE models suffer from several shortcomings jeopardizing their ability to predict, let alone to prevent, a global crisis:

• Agents are assumed to be perfectly rational, having perfect access to information and adapting instantly to new situations in order to maximize their long-‐run personal advantage

• So far agents have entered the models as homogeneous representative entities, while it would be a step forward being able to take into account agents heterogeneity

• Canonical models consider atomistic agents with little or no interactions and thereby are not able to cope with network externalities

But most of all it is the very notion of equilibrium which prevents standard models from dealing with crisis. A stable steady state equilibrium is a condition according to which the behaviour of a dynamical system does not change over time or in which a change in one direction is a mere temporary deviation. This condition is proper of general equilibrium theory, in which a stable steady state is believed to be the norm rather than the exception. When in the canonical model we are out of equilibrium, the situation is seen just as a short lapse before the return to the steady state. This is in sharp contrast with the very notion of crisis, which represents a steady deviation from the equilibrium. Loosely speaking, the crisis phenomenon is not even conceived within the framework of standard models.

All these flaws are not only related to DSGE models, but also to Computational General Equilibrium (CGE) or macro-‐econometric forecasting models, which are the traditional policy making tools. In this view it would be very important to find new frameworks capable of avoiding those shortcuts.

12 http://www.ecb.int/home/html/researcher_swm.en.html


26 | P a g e

Some of such methodologies and methods already exists and some governments are using them. Our aim is to push forward in that direction.

We need to move away from the equilibrium paradigm in order to be able to assess other issues: evolutionary dynamics; heterogeneity of technologies and firm; political and legal determinants of social stability; incentive structures; better modelling technological change, innovation diffusion and economic systems (taking into account finance, debt and insurance); interactions between heterogeneous economic agents (firms and households) and central governments; heterogeneous responses to government incentives; economic dependence from the ecosystem.

Trichet, the former head of ECB, clearly put it: “This doesn't mean we have to abandon DSGE...(but)...atomistic rational agents don't capture behaviour during a crisis...rational expectations theory has brought macroeconomics a long way ... but there is a clear case to re-‐examine the assumptions”

But the need for new policy making tools is not limited to the economic realm: in the future it will become more and more important to anticipate non-‐linear potentially catastrophic impacts from phenomena such as: climate change (draught and global warming); threshold climate effects such as poles’ sea-‐ice withdraw, out-‐gassing from melting permafrost, Indian monsoon, oceans acidification; social instability affecting economic well-‐being (social conflict, anarchy and mass people movements).

The lack of understanding of systemic impact has driven to short term policies which failed in grasping long term, systemic consequences and side effects:

-‐ An example of this approach might be given by the sovereign debt issue. In fact it is relatively easy for governments under popular pressure to increase expenditure and public debt to cope with short term necessities, such as offsetting the negative impacts brought about by a regional or global crisis. On the other hand it is harder to take into account the long term effect determined by higher interest rates on private investments and consumption through crowding out and fiscal pressure.

-‐ Another example of short-‐termism are the financial policies pursued in south East Asia at the beginning of the 90s. Many countries, such as Thailand, liberalized their financial markets fostering the inflow of investments aimed at sustaining growth. Unfortunately those capitals triggered a real estate bubble which has been at the roots of the 1997-‐1998 crisis.

-‐ In 2008 the Central Bank of Iceland yielded liquidity loans for saving banks on the verge of default on the basis of newly-‐issued, uncovered bonds, i.e. effectively printing fiat money on demand, causing a significant rise in inflation. To cope with this rise in prices, the Iceland Central Bank had to keep very high interest rates thereby leading to an economic bubble.

-‐ According to a large number of economists the financial crisis was triggered by US government policies spanning across two administrations which were intended to ensure citizens’ right but instead determined an unprecedented high number of risky mortgages, as well as the decline in mortgage underwriting standards that ensued. According to the “Financial Crisis Inquiry Commission Report” 13 those policies, together with the deregulation of the financial system, might have catalysed the crisis.

13 http://fcic.law.stanford.edu/report


27 | P a g e

-‐ Other examples can be the bail out of financial institutions: in the short run those actions maintain employment and economic standards, while in the long term they induce moral hazard, keep operating inefficient companies and decrease the trust of economic agents in regulation, which is the funding pillar of our economic system.

2.3.5. Ensure long -‐ term thinking

In traditional economics, decisions are utility-‐maximising. Agents rationally evaluate the consequences of their actions, and take the decision that maximize their utility. However, it is well known that this rationalistic view does not fully capture human nature. We tend to overestimate short-‐term impact and underestimate the long term. In policy-‐making, short-‐termism is a frequent issue. People are reluctant to accept short-‐term sacrifices for long-‐term benefits. Politicians have elections typically every 5 years, and often their decisions are taken to maximize the impact “before the elections”. There is also the perception that laypeople are less sensitive to long term consequences, which are instead better understood by experts. Overall, long-‐term impact is less visible and easier to hide, due to lack of evidence and data. As a result, decisions are too often taken looking at short-‐term benefits, even though they might bring long term problems.

Climate change is a typical policy area where sub-‐optimal decisions were taken because the short-‐term costs were considered to outweigh the long term consequences. The long term impact is not visible, while the short term sacrifices were, even though ICT had an important role in stimulating the debate and catalysing attention of the media on the issue.

2.3.6. Encourage behavioural change and uptake Once policies are adopted, a key challenge is to make sure that all stakeholders comply with regulations or follow the recommendations. It is well known how the greatest resistance to a policy is not active opposition, but lack of application.

For instance, several programmes to reduce alcohol dependency problems in the UK failed as they excessively relied on positive and negative incentives such as prohibition and taxes, but did not take into account peer-‐pressure and social relationships. They failed to leverage “the power of networks” (Ormerod 2010). For instance, any policy related to reduction of alcohol consumption through prohibitions and taxes is designed to fail as long as it does not take into account social networks, as binge drinkers typically have friends who also have similar problems. In another classical example (Christakis and Fowler 1997), a large scale longitudinal study showed that the chances of a person becoming obese rose by 57 per cent if he or she had a friend who became obese.

The identification of social networks and the role of peer pressure in changing behaviour is not considered in traditional policy-‐making tools.

2.3.7. Manage crisis and the “unknown unknown”

The job of policy-‐makers is increasingly one of crisis management. There is robust evidence that the world is increasingly interconnected, and unstable (also because of climate change). Crises are by definition sudden and unpredictable. Dealing with unpredictability is therefore a key requirement of policy-‐making, but the present capacity to deal with crises is designed for a world where crises are exceptional, rather than the rule. Donald Rumsfeld, former secretary of state, famously said during the Iraq war that while the US government was capable of dealing with the “known unknown”, the difficulty was the increasing recurrence of “unknown unknown”: those things that we don’t known that we don’t know.

There is evidence that the instability and chaotic natures of our world is increasing, because of its increasing connectedness. Every year, intense climate phenomena throw our cities in disarray,


28 | P a g e

because of snow, flooding, fires. Each crisis seems to find our decision-‐makers unprepared and unable to deal with it promptly. As Taleb (2007) puts it, we live in the age of "Extremistan": a world of "tipping points" (Schelling 1969) “cascades” and "power laws" (Barabasi 2003) where extreme events are "the new normal". There are many indications of this extreme instability, not only in negative episodes such as the financial crisis but also in positive development, such as the continuous emergence of new players on the market epitomised by Google. The random vulnerability of today’s world is well illustrated by this chart from the EC DG RESEARCH.

Figure 4: Total Disasters Reported

2.3.8. Moving from conversations to action

The collaborative action of people is able to achieve seemingly unachievable goals: experiences such as ZooGalaxy and Wikipedia show that mass collaboration can help achieve disruptive innovation. Yet too often web-‐based collaboration is confined to complaints and discussions, rather than action. As one blogger put it, paraphrasing Marx: “Philosophers have only interpreted the world: the point is to complain about it”14.

For example, the 2012 Italian elections saw an explosion of activity in social media discussing about the different candidates. This energy then failed to translate into concrete action in the aftermath of the elections.

2.3.9. Detect non-‐compliance and mis-‐spending through better transparency

In times of crisis, it is ever more important for governments to ensure that financial resources are well spent and policies are duly implemented. But monitoring is a cost in itself, and a certain margin of inefficiency in resources deployment is somehow “natural”. Yet the cost of this mismanagement is staggering: for instance, in 2010, 7.7% of all Structural Funds money was spent in error or against EU rules15. OECD estimates place the cost of corruption equals 5% of global GDP16. Thereby it would be crucially important to be able to avoid the mismanagement with anticipatory corrective actions.


29 | P a g e

2.3.10. Understand the impact of policies

Measuring the impact of policies remains a challenge. Ideally, policy-‐makers would like to have real-‐time clear evidence on the direct impact of their choice. Instead, the effects of a policy are often delayed in time; the ultimate impact is affected by a multitude of factors in addition to the policy. Timely and robust evaluation remains an unsolvable puzzle.

This is particularly true for research and innovation policy, where the results from investment are naturally expected at years of distance. As Kuhlmann and Meyer-‐Krahmer (1994) puts it, “the results of evaluations necessarily arrive too late to be incorporated into the policy-‐making process”.

2.4. When policy-‐making 2.0 becomes a reality: a tentative vision for 2030

This is the scenario of how future policy-‐making could be deployed in an ideal world, if all the opportunities of policy-‐making 2.0 tools were taken. It aims at illustrating how these technologies and methods could concretely be deployed and the effect they would have. This is deliberately a normative scenario, describing a positive and concrete future at a very high level.

The scenario is organised alongside the typical phases of the policy-‐making cycle. It applies to a hypothetical new privacy directive being developed in 2030.

2.4.1. Agenda setting phase: recognizing the problem

Brussels, 2030. The EC task force on privacy and data protection is alerted by a number of events. Their yearly report, accompanied by the publication as linked open data in February, has been accessed by more than 10.000 people in a week. Several high profile online blogs have published the geo-‐visualized mash up of the task force data with the data from customer complaints about broadband slowness. The figures speak for themselves: the complaints from customers, collected through both the government single feedback system and social media, about privacy infringements and identity theft mirror exactly the broadband disruption. All seem to point to some kind of "data theft" at the infrastructural level of the Internet. A similar analysis on open linked data shows an abnormal concentration of complaints over credit card fraud from users of a limited number of ISP that have struggled to obtain the infrastructure security certification. Anomalies in this correlation seem to weaken the case, but are quickly discovered when social network analysis is carried out: not only the users of ISP but also their friends and contacts are most likely to report denounces for fraud.

While some years ago the task force members would still address this through the traditional slow policy process, only to realize its social impact after mass media take this up, today a quick look at social media analytics confirms that the public is deeply concerned. Hashtags like #wherearemydata are drawing thousands of comments. The task force obtains real time report on sentiment and opinions being shared publicly; it appears clear that people feel unprotected by existing instruments and regulation and voice their dissatisfaction mainly towards the Task Force itself. In particular, the

14 Quoted in Mick Fealty, The wisdom of crowds, The Guardian 24 February 2007 http://www.guardian.co.uk/commentisfree/2007/feb/24/towardsadeliberativedemocra 15 http://www.europeanvoice.com/article/2011/november/commission-‐names-‐worst-‐managers-‐of-‐eu-‐money/72613.aspx 16 http://www.oecd.org/dataoecd/51/5/49693613.pdf


30 | P a g e

reputation report quickly identifies a limited numbers of social media activists that show high influence in terms of shaping the public opinion on the matter, as their message is quickly spread.

Historical text analysis of social media allows to predict that users that complain over privacy infringement are likely to dramatically decrease the extent to which they share information and data on the web over the following weeks. This drastic cuts to content sharing becomes a serious liability for an economy which is now built on the assumption that people naturally share and collaborate on the web. Reduction in knowledge sharing, as predicted by social media analytics, could lead to a reduction of economic activity which is already fragile.

An in-‐depth investigation discovers a hardware hacking group that has targeted a selected number of lowly protected broadband providers to steal data directly from their traffic. The policy agenda of the Task Force are quickly switched to develop a revision of regulation in order to better link the broadband regulation with data protection.

An open collaboration group is convened, with the direct involvement of users previously identified as "highly influential" through social media analytics. In addition, cross-‐analysis reputation tools are used to identify "experts" on joined-‐up policy approaches to data protection and broadband infrastructure, based on integrated data from social media (e.g. Klout and Linkedin) and scientific impact (e.g. Altmetrics, ISI impact factor). This group is called to provide independent fact based analysis of the problem based on best available data.

In particular, the group is called to understand and model the causal relationship between fear of privacy infringements and reduction in knowledge sharing; and between this reduction and economic growth. The analysis is carried out through a combination of network analysis, system dynamics and agent based modelling. Their report simulates several possible scenarios, but the common theme is that a reduction in sharing activities by key influencers could lead to a major economic downturn, as non-‐sharing behavior will soon spread from the "geeks" to the general public through imitation and social pressure.

The report is published for public review, enabling in-‐line comments and in-‐depth analysis of the raw data and models behind the analysis. It brings in hundreds of comments. Once a quick text mining analysis is carried out, the comments seem to cluster on an unjustified assumption in one scenario, and on a limited set of issue regarding the potential negative impact on net neutrality when implementing new regulation. Hence, the scenario is double-‐checked and the assumption clarified, and net neutrality experts are brought on board in the working group.

2.4.2. Policy design Once the nature and size of the problem is clarified, the working group is called to design possible policy measures. A crowdsourcing exercise is launched, where anyone can submit ideas for specific amendments to the present regulation. The analysis is based on the most voted suggestion, but these turn out to be not extremely insightful. A reputation management system is integrated with the exercise, allowing to identify original ideas and insight based on voting “weighted” based on expertise in the field. A set of 10 recommendations are presented for further analysis by the working group.

The working group, based on this input, formulate 3 policy options alongside these axis:

-‐ continuing with the current data protection framework

-‐ enhancing it with greater forms of self-‐regulation; increased transparency, easier enforcement and greater empowerment of users

-‐ define a new, stricter data protection regulation


31 | P a g e

The three options are then run through the large-‐scale simulation engine, which combines agent based modeling, system dynamics, network analysis and big data analytics. This allows to anticipate the unexpected effect of a new stricter regulation, that would probably induce virtuous broadband providers to conform to the minimum requested by regulation, while private companies that typically choose the most secure providers would have to increase their expenditure on security, in particular in the field of web services, which is already weak in Europe and exposed to global competition. In addition, consumers would be satisfied with a perception of increased security, thereby reducing their attention on own control over data, which would in the long term increase the security risks of another crisis.

All the models underlying the simulation are open to public review, in order to ensure the transparency of the initiative. Indeed, the first version of the ex-‐ante impact assessment showed that option 1 was the less risky and more beneficial, but a little known researcher from Greece quickly pointed out to a banal coding mistake in the database used to compute historical series of privacy infringements.

In the end, option 2 is chosen as the most effective. The amendment to the regulation are quickly rapidly drafted, publicly reviewed and then turned into law by the European Parliament.

2.4.3. Implementation

The regulation envisages a strong role for the public in both enforcement and self-‐regulation. Each local branch of broadband providers have to publish in real time as open linked data the results of the security certification, as well as any traffic management intervention they carry out.

In addition, a set of “persuasive games” has been developed to help consumers manage and control their data flows. Users receive badges each time they perform a data safety self-‐assessment, which is easy to carry out through a highly visual smartphone app which highlights to what extend the users behaviour diverges from the public recommendations and from the people in his/her social network. Unexpectedly, a kind of game about being “safest kid on the block” starts particularly between teenagers, that compete in trying to overcome each other safety provisions. New business models of third party data management services are launched for those less interested in managing their data.

As a result, the propensity to buy, share and collaborate online increase sensibly, driving to a moderately positive economic impact.

2.4.4. Evaluation Real-‐time data analytics on the performance of data providers, as well as anonymized data on consumers data protection measure, allow decision-‐makers to identify potential breaches as soon as they happen.

Participatory sensing tools, combined with opinion mining allow citizens and policy-‐makers to easily monitor when new problems emerge.

Open data on measures taken by regulators allow civil society organisation to ensure the adequacy of government intervention.


32 | P a g e

2.5. The key challenges for policy makers and the corresponding phases in the policy cycle

Let us now relate the key challenges of policy making activity with the phases in the policy cycle: • The Agenda Setting phase is mostly related to the challenges

o Detect and understand problems before they become unsolvable

o Manage crisis and the “unknown unknown” o Ensure long -‐ term thinking: Agenda

• The Design phase is mostly related to the challenges

o Encourage behavioural change and uptake o Identify “good ideas” and innovative solutions to long-‐standing problems

o Reduce uncertainty on the possible impacts of policies o Generate high involvement of citizens in policy-‐making

• The Implementation phase is mostly related to

o Moving from conversations to action o Reduce uncertainty on the possible impacts of policies

• The Monitor and Evaluation phase is mostly related to

o Detect non-‐compliance and mis-‐spending through better transparency o Manage crisis and the “unknown unknown”


33 | P a g e

3. The supply side: current status and the Research Challenges

In this section, we illustrate in detail each research challenge which needs to be addressed in order to make the vision a reality and address the policy-‐challenges described in the previous chapter, by describing:

-‐ The definition,

-‐ The potential opportunities for governance,

-‐ The state of the art of market and research,

-‐ The existing challenges and

-‐ The recommended research themes.

The research challenges are organised in 2 groups: the first regroups 6 challenges on Policy Modelling, while the second one regroups 9 challenges on Collaborative Governance.

3.1. Policy Modelling

3.1.1. Systems of Atomized Models

Introduction and definition

This research challenge seeks to find the way to model a system by using already existing models or composing more comprehensive models by using smaller building blocks, sometimes also called “atoms”, either by reusing existing objects/models or by generating/building them from the very beginning. Therefore, the most important issue is the definition/identification of proper (or most apt) modelling standards, procedures and methodologies by using existing ones or by defining new ones. Further to that, the present sub-‐challenge calls for establishing the formal mechanisms by which models might be integrated in order to build bigger models or to simply exchange data and valuable information between the models. Finally, the issue of model interoperability as well as the availability of interoperable modelling environments should be tackled, as well as the need for feedback-‐rich models that are transparent and easy for the public and decision makers to understand.

Why it matters in governance

Using existing objects/models that are able to describe systems, sub-‐systems and interaction among them, allows everyone to build his own insight on a specific problem/solution. So, in governance, such opportunity gives us the chance to:

• Release public data, linking them and producing visual representations able to reveal unanticipated insights.


34 | P a g e

• Use social computing to promote engagement and citizens’ inclusion in policy decision, and exploit the power of ICT in mining and understanding the opinions they express.

• Analyse policies and produce models that can be visualised and run to produce simulations able to show the effects and impacts from different perspectives such as political, economic, social, technological, environmental and legal facets.

Current Practice and Inspiring cases

In systems analysis, it is common to deal with the complexity of an entire system by considering it to consist of interrelated sub-‐systems. This leads naturally to consider models as consisting of sub-‐models. Such a (conceptual) model can be implemented as a computer model that consists of a number of connected component models (or modules). Component-‐oriented designs actually represent a natural choice for building scalable, robust, large-‐scale applications, and to maximize the ease of maintenance in a variety of domains.

An implementation based on component models has at least two major advantages:

• First, new models can be constructed by coupling existing component models of known and guaranteed quality with new component models. This has the potential to increase the speed of development.

• Secondly, the forecasting capabilities of two different component models can be compared, as opposed to compare whole simulation systems as the only option.

Further, common and frequently used functionalities, such as numerical integration services, visualisation and statistical ex-‐post analyses tools, can be implemented as generic tools and developed once for all and easily shared by model developers. By the way, the current practice in composing and re-‐using models is still not sufficiently widespread. In relation to Model Reuse, this is mainly due to the fact that little to no repository actually exists17. Moreover, the publicly available models are not “open” to modification or re-‐use. It would be useful if every paper containing a model included a link to on-‐line version that people could run and modify, Some modelling environments (or modelling suites) provide some examples and small libraries of ready-‐to-‐use models, but in most cases, they are not completely open nor any explanation is provided on how to reproduce them (their structure, parameters, etc.). As an inspiring case see the SEAMLESS project, which was funded by the EU Framework Programme 6 (Global Change and Ecosystems), ran from 2005 till March 2009, and developed a computerized framework for integrated assessment of agricultural systems and the environment18. During the project, a modular approach was chosen to develop a system named “Agricultural Production and Externalities Simulator (APES)”, illustrated in figure (5). APES is a modular simulation system targeted at estimating the biophysical behaviour of agricultural production systems in response to the interaction of weather, soils and different options of agro-‐technical management. Although a specific, limited set of components is available in the first release, the system is being built to incorporate, at a later time, other modules which might be needed to simulate processes not included in the first version. The processes are simulated in APES with deterministic approaches which are mostly based on mechanistic representations of biophysical processes. APES was used to compare alternative agricultural and environmental policy options, facilitating the process of assessing key indicators that characterize interactions between agricultural systems, natural and human resources, and society. The developed framework, named SEAMLESS-‐IF

17 This is true for most of the sectors, even though for instance most energy models are based on the MARKAL family of models. Furthermore something to consider is that models need to be customized, so that having a single framework readily applicable to different contexts and sectors may actually be counter productive 18 http://www.seamless-‐ip.org


35 | P a g e

in a finale stage, also enabled linkage of quantitative models, pan-‐European databases and qualitative procedures to simulate the impact on society of biophysical, economic and behavioural changes. SEAMLESS-‐IF now facilitates ex-‐ante assessments at the full range of scales from the global to the field level to support policy and decision making for sustainable development. SEAMLESS-‐IF nowadays can be used to investigate the effects of agricultural and environmental policies while accounting for technical innovations. Further, the interactions of such policies with other major trends such as climate change and increasing land used for bio-‐fuel crops can be studied efficiently in the near future.


36 | P a g e

Figure 5: Agricultural Production and Externalities Simulator (APES)

Analyses with SEAMLESS-‐IF can be done at multiple scales and with varying time horizons, whilst focusing on the most important issues emerging at each scale. This is possible as the framework is based on research innovations in linking models across scales allowing consistent “micro-‐macro” analysis as well as linking models across disciplines allowing “economicbiophysical” analysis. The linked models range from a bio-‐physical field model to a farm model and to an agricultural sector model for the EU; in other words they ensure a consistent analysis of what effects EC policies may have on agricultural markets, farming systems and the environment. In addition, the effectiveness of a policy in its institutional context is assessed by applying qualitative procedures. The interlinked pan-‐European database provides the relevant data needed at different scales.

For another inspiring example have a look at the Insight Maker case at http://insightmaker.com/. Insight Maker allows to build simulation models ("Insights") for all scales: from the smallest cell, to the social effects of product adoption, to global climate change. Once they are built one can share them with others. The models are called “an Insight” as they will typically reveal one or more fascinating point about the system under study. All the simulations built with Insight Maker can be shared via the web. This means people can change the variables and see the results for themselves.

Vensim Molecules19 is a software used for constructing system dynamics models from molecules of system dynamics structure. Molecules are made of primitive stock and flow or auxiliary elements and are, in turn, the building blocks of complete models, elements of substructure serving a particular purpose. Molecules provide a framework for presenting important and commonly used elements of model structure making faster and easier to develop system dynamics models.

19 http://www.vensim.com/molecule.html


37 | P a g e

Anylogic20, a multi-‐method simulation modelling tool capable of integrating and combining the following modelling approaches: system dynamics, discrete event simulation and agent-‐based modelling. Anylogic’s simulation language is composed by stock and flow diagrams (used for System Dynamics modelling), statecharts, which define the agents’ behaviour in Agent Based modelling, action charts (used to define algorithms), and finally process flowcharts which are the basic constructions for defining processes in Discrete Event modelling.

Available Tools

A very interesting tool is En-‐ROADS21, a global simulation model that focuses on how changes in global GDP, energy efficiency, R&D results, carbon price, fuel mix, and other factors will change carbon emissions, energy access, and temperature.

En-‐ROADS is designed to complement, other more disaggregated models addressing these questions, and relies on the other models and EIA projections for testing and data.

20 http://www.xjtek.com/

21 http://climateinteractive.org/simulations/en-‐roads


38 | P a g e

En-‐ROADS is customized to address enquiries regarding how much might technological breakthroughs contribute to addressing climate change. These particular breakthroughs include for instance R&D and scale-‐up of a new zero-‐carbon energy supply, renewable energy, energy efficiency, inexpensive natural gas, etc. More precisely En-‐ROADS investigates which assumptions about the technology and the economy would be necessary for a breakthrough to grow with enough speed and scale to deliver climate goals.


39 | P a g e

One of the most innovative parts of En-‐ROADS regards its capabilities to test assumptions about the potential success of R&D towards zero-‐carbon energy. More in particular the simulation investigate what are the likely dynamics of the emergence of a new energy supply, as well as how fast could it grow and displace high-‐carbon sources and reduce carbon emissions. En-‐ROADS is an extension of the C-‐ROADS model, which will be described below in the roadmap. The distinction between the two models is that while C-‐ROADS focuses on how the changes in national and regional emissions could affect GHG emissions and climate outcomes, En-‐ROADS focuses on how changes in the energy, economic, and public policy systems could influence GHG emissions and climate outcomes.

Key challenges and gaps

With regards to implementation architecture and use of modelling frameworks, there are two major problems:

• the framework design and implementation must be optimized to balance carefully its flexibility and its usability to avoid incurring either a performance penalty or users having too steep a learning curve, and

• developing components for a specific framework constrains their use to that framework.

The most immediate option to overcome such problems is developing inherently reusable components (i.e. non framework specific), which can be used in a specific modelling framework by encapsulating them using dedicated classes called “wrappers”; such classes act as bridges between the framework and the component interface. The disadvantage of this solution is the creation of another “layer” in the implementation, which adds to the already implemented machinery in the framework. The appropriateness of this solution, both as ease of implementation and overall performance, must be evaluated case by case.

Regardless of the choice of developing framework specific or intrinsically reusable components, there is a basic choice which must be carefully evaluated prior to that and which is related, in general terms, to the framework as a flexible modelling environment to build complex models


40 | P a g e

(model linking), but also to the framework as an efficient engine for simulation, calibration and simulation of model components (model execution). Modern software technologies allow building flexible, coherent and elegant constructs, but that comes at a performance cost. Without even introducing specific references to Object Oriented Programming (OOP), it seems important to point out that the use of object-‐oriented programming constructs, which actually enhance flexibility, modularity and reuse of software, all nice things, require the compiler to use virtual methods calls, dynamic dispatching, and so on. All these operations are resource intensive and in some cases, they can heavily affect the code performance, and this becomes evident in applications in which such use is done thousand times every simulation step.

By the way, the Model Composition horizon is even more clouded as the potential advantages resulting from the possibility of composing bigger models from smaller ones have been shown only recently. It is essentially due to the problem of interoperability and integration of different vendors’ (thus proprietary) model formats and to the lack of standards allowing performing composition tasks. Another problem stems from the fact that many models are still too dependent on their implementation methodology. Moreover, model integration is at present almost non-‐existing. Very few modelling environments/suites provide the import/export functionalities and a standard language for model interoperability is not currently available. Most of the current practice for data communication or information transfer is performed by means of third party solutions (e.g.: interoperability in most cases is achieved by transferring data via electronic spreadsheets or, only in rare cases, by using Database Management Systems (DBMS) or Enterprise Resource Planning (ERP) systems.

Current research

Current research, as well as previous research, has not yet worked on (with the exception of just a few cases) the problem of different models integration. At present, due to the plethora of different modelling/simulation environments/suites, as well as to differences at the scientific field level, many competing file formats exist. It is possible that vendors perceive the modelling practice as a very small market niche (as the users stem mainly from Academia and to a very small extent from private companies where a Decision Support Systems is used, what is more the Public Administration share is negligible) and therefore are reluctant to introduce interoperable features.

Also, current research, as well as previous research, has only recently begun to explore the following issues:

• Open-‐source modelling and simulation environments (there are open environments that are rising in importance in the research community, albeit in most cases they only provide the possibility to implement and simulate a model according to the modelling methodology they refer to).

• Communication of data among models developed in different proprietary (or open) environments by depending on third party solutions (e.g.: interoperability is in most cases only achieved by transferring data by means of electronic spreadsheets or, only in rare cases, by using a DBMS or an organisation’s ERP).

• Open visualisation of results stemming from model simulation (e.g.: online visualisation of simulation results in a browser by interfacing -‐ only in a few cases -‐ the simulation engines, or -‐ as it is more often the case -‐ by connecting to a third party mean, as described in the previous bullet point).


41 | P a g e

Future research

Future research should therefore focus on:

• Definition of standard procedures for model composition/decomposition, e.g. how to deductively pass from a macro-‐description of models to the fine definition of its building-‐blocks or molecules (top-‐down approach), how to inductively conceive a progressive composition of bigger models by aggregating new modules as soon as they are needed (bottom-‐up approach) or by expanding already existing objects.

• Proposition of a minimum set of archetypical structures, building blocks or molecules that might be used according to the proper level of decomposition of the model (e.g. systemic archetypes, according to the Systems Thinking / System Dynamics approach, might be useful to describe the overall behaviour thanks to the main variables in the system to be modelled at a macro-‐to-‐middle level). The procedures to implement, validate and redistribute any further improvement of these “minimal” objects should be investigated.

• Definition of open modelling standards, as the basis for interoperability, that is defining common file formats and templates (i.e.: by means of XML), which would allow the models described by means of these XML files to be opened, accessed and integrated into every (compliant) model-‐design and simulation environment22.

• Interoperability, also intended in terms of Service Oriented Architectures (e.g.: certain stand-‐alone and always operative models might expose some “services” in order to make available either their endogenous data or bits of information, or some peculiar function or structural part, while some other may request to use those services when needed. In consequence, it creates a need for a definition of model repositories, a list of operative models and the functionalities that they might expose which finally, entails the definition of a SOA among interoperable models).

• Definition and implementation of model repositories (and procedures to add new objects to them), even if they are restricted to hosting models developed according to a specific methodology (Agent Based, System Dynamics, Event Oriented, Stochastic, etc.)

• Definition and implementation of new relationships that are created when two models are integrated. All possible important relationships resulting from a model integration/composition should be identified and eventually included in the new deriving integrated model.

• Input / Output definition / re-‐definition: the integration of modelling techniques is a pertinent issue in the scope of this challenge. The multi-‐modelling tools should be, in the future, available not only to experts but also to lay users. Moreover, at present, only a few of the actually available modelling/simulation suites are able to provide the possibility to build a model by referring to a different modelling methodology.

22 Making portability is very hard to implement: in the past there has been a significant effort with SMILE and later XMILE to make SD models portable between different software programs. This was really a best case scenario as the software programs in play were so very similar to start, but it has ended up unsuccessfully as agree and implementation has not been reached. Deciding on a universal format to bind existing things together seems like it will not be able to work (http://xkcd.com/927/). There has been greater success with things like Modelica where you get a format and an app and then other programs decide to adopt the format themselves, a more organic process


42 | P a g e

3.1.2. Collaborative Modelling


The English sayings “two heads are better than one” and “too many cooks spoil the broth” give an idea of the expectations that arise from a collaboration of people. On the one hand, one would expect that a group of people is able to better observe and perceive situations as well as to make better decisions than a single person would be able to make. On the other hand, it is also common knowledge that the collaboration of several people entails the problem of group coordination, which, if disregarded, can make group work inefficient, compared to the work of a single person. There is the need for an authoritative gatekeeper that is also the modeler implementing the model, otherwise collaborative sessions would get out of control quickly with non-‐implementable ideas becoming the focus of discussion very fast.

There are three kinds of problems that are typically approached by groups:

• cognition problems, problems with a definite solution or a set of solutions that are certainly better than others;

• coordination problems, problems that require the group to figure out how to coordinate the behaviour of its members;

• cooperation problems, problems which feature the involvement of several self-‐interested, distrustful people who have to work together.

Collaborative modelling (also called group model building) refers to a process where a number of people actively contribute to the creation of a model. The weakest form of involvement is feedback to the session facilitator, similar to the conventional way of modelling. Stronger forms are proposals for changes or (partial) model proposals. In this particular approach the modelling process should be supported by a combination of narrative scenarios, modelling rules, and e-‐Participation tools (all integrated via an ICT e-‐Governance platform): so the policy model for a given domain can be created iteratively using cooperation of several stakeholder groups (decision makers, analysts, companies, civic society, and the general public).

As a matter of fact groups require rules (or cultural norms) to maintain order and coherence, as well as diversity and independence of its group members in order to create a kind of a collective intelligence. Bringing together people with diverse perspectives and backgrounds for working together in multi-‐disciplinary teams is expected to improve the overall group performance, so the first issue on which the collaborative process should be based is the definition of a shared modelling rules framework (the social norms), guiding the modelling team in determining whether a proposal is accepted or rejected. Two usually adopted types of rules are:

• Rules of majority, where a certain number of group members had to support or oppose a proposal in order for the whole group to accept or reject it (e.g., more than half). A tie-‐break rule was sometimes specified (e.g., for the case of an equal number of supporters and opponents). The tie-‐break could involve seniority issues.

• Rules of seniority, where the weight of a group member’s support or opposition was related to his or her status within the group. This status could be acquired (e.g., by experience) or associated with a position to which the member was appointed. A frequent example of this was the case of a more experienced modeller who was considered as the leader by the group and took decisions on their behalf. The other members filled the role of consultants in such a case.


43 | P a g e

These rules were sometimes set up explicitly before the group began their work, or in an early phase of this work. But in most cases they rather emerged as the result of each member’s behaviour. Individuals making regular contributions of high quality were likely to acquire seniority. In homogeneous teams majority rules were used more often.


From a very high level of abstraction, collaborative modelling itself can be seen as a social interaction between several people, while these people who together perform the modelling process form a social entity. Thus, the process of collaboratively defining and implementing a model, with a particular reference to the public policy modelling, is strictly connected with the public aspect of every citizen’s life, starting from the communities bridged by the decision makers that collaboratively define some policies, to an average citizen which interacts with other citizens within the rules framework defined by the policies themselves.

Starting from the needs perceived by the citizens, the limitations of existing modelling techniques adopted in policy making include the following issues:

• Changing models is too time-‐consuming and integrating to other diagrams is difficult. Also there are version control problems.

• It is not possible for more than one person to work on the same diagram at the same time.

• Modelling has to be done at the specific location where the modeller is present.

• Contribution to the model comes from those interviewed or at a group meeting, limiting the potential contribution from a larger group.

• Low model acceptance: the model resulting from the modelling session is not supported by some of the stakeholders.

• Participants feel misunderstood: as a consequence of bad elicitation or a wrong understanding of the model.

• Low perceived model quality and limited model comprehension: Individuals do not fully understand the model or do not agree with it.

Reasons that argue for conducting policy modelling in a collaborative manner are:

• No person typically understands all requirements and understanding tends to be distributed across a number of individuals.

• A group is better capable of pointing out shortcomings than an individual.

• Individuals who participate during analysis and design are more likely to cooperate during implementation.


44 | P a g e

Collaborative modelling calls for the definition of the citizen’s role in the public policy modelling process (e.g.: the mass participation issues and processes have been already researched in depth by the e-‐Participation research programs). In order to guarantee participation there are some prerequisites that should be fulfilled:

• All citizens who access ICT services in order to participate should represent the views of communities affected by the given policy;

• All citizens are able to take part in the modelling process via intuitive IT systems that enable them an effective and efficient contribution;

• All citizens possess proper skills (or are assisted) to purposely follow a process of group model-‐building in order to avoid/abate wrong mental models and thus ultimately reach a shared vision of the problem.


In current practice, collaborative modelling is mainly performed offline; still the rules and guidelines for session processes are not yet sufficiently widespread. In fact, the abatement of wrong mental models and the creation of knowledge from information usually imply the dialogue among people with different views of the problem as well as the need for critical skills. Further to that, the information that occurred in a discussion has to be grounded and definitively transferred to the formal model. Thus, e-‐Participation might be of help in achieving a critical mass of data and information exchange online but in itself does not solve the problem of mass cooperation and collaboration in a formal modelling process. Even more, the participation in this process entails, at present, a thorough knowledge on modelling processes or tools that an average citizen does not have. Therefore, there is an urgent need for Intuitive Interfaces, Modelling Wizards and guided simplified approaches to modelling. Starting from the relevance of collaborative modelling in policy making, as a very former inspiring case Maarten Sierhuis and Albert M. Selvin, working at NYNEX Science & Technology Inc in New York, presented in 1996 a applied research report on “Towards a Framework for Collaborative Modelling and Simulation”, describing methodologies for modelling and simulation in a collaborative analysis or design project, and describing a case study in which Conversational Modelling, a software-‐supported technique for collaborative modelling, enabled participants to construct static knowledge models in collaborative sessions. The sessions described in the report resulted in the identification of 207 queries. Of these, 24 were chosen for detailed modelling. As a result of the modelling, 44 resources, 29 knowledge items, 58 data items, and 8 organizational issues were identified. The response from participants was positive. Many stated that they had learned more about each other work in the conversational modelling sessions then they had been able to in the course of their normal work activities. The development organization has been able to use the output of the sessions to generate design requirements. A picture of the interface (figure 6) used during the sessions follows.


45 | P a g e

Figure 6: Conversational Modelling Interface

As more recent inspiring case, one can refer to the results of the FP7 projects OCPOMO23 or PADGETS24. This last one, PADGETS, aims at bringing together two well established domains, the mashup architectural approach of web 2.0 for creating web applications (gadgets) and the methodology of system dynamics in analysing complex system behaviour. The objective is to design, develop and deploy a prototype toolset that will allow policy makers to graphically create web applications that will be deployed in the environment of underlying knowledge in Web 2.0 media. The project introduces the concept of Policy Gadget (PADGET) – similarly to the approach of gadget applications in web 2.0 – to represent a micro web application that combines a policy message with underlying group knowledge in social media (in the form of content and user activities) and interacts with end users in popular locations (such as social networks, blogs, forums, news sites, etc.) in order to get and convey their input to policy makers.

23 Open Collaboration in Policy Modelling, http://www.ocopomo.eu 24 http://www.padgets.eu


46 | P a g e

Figure 7: the PADGET Framework

PADGET is composed of four main components:

• A message, that is a policy in any of its stages and forms

• A set of interaction services, that allows users to interact with the policy gadget (find it, access its content, comment its content, share it etc.). These interfaces may be provided by either the underlying social media platforms in which the PADGET Campaign is launched or by the PADGET itself when it takes the form of a micro application (i.e. in the case of the iGoogle gadget).

• The social context, that is the framework describing the social activity and content relating with the policy gadget in each individual social media platform where the policy gadget is present.

• The decision services, which are offered by two modules. The PADGETS analytics and the PADGETS simulation model. The decision services component is responsible for the generation of the information outputs to be presented to the PADGET initiator (usually a policy maker).

PADGETS will use publicly available APIs for interconnecting, publishing and retrieving content from underlying social media platforms. The collected information and user activities that policy gadgets invoke in the media platforms will be categorized using semantic tags as to their relation to the policies in order to help the policy maker form an opinion about what the users think about relevant issues and policies.


47 | P a g e

An interesting model is Threshold 21 (T21), built using the System Dynamics methodology to facilitate decision making. Feedback relations among key variables within sectors are endogenously simulated by T21, and those relations can lead to further feedback to other sectors. This process is valuable for learning more about the complex interactions among and across sectors that need to be taken into account in order to develop more effective policies and mitigate or avoid negative side effects. This approach allows to bring experts together from different sectors to better understand these relations and obtain the data needed to translate a qualitative causal diagram (a map of the system) into a quantitative model, using a participatory approach.

Source: “Macro Economic Policy Analysis Applications”, presented by John Shilling at the Transatlantic Research on Policy Modelling Workshop25

An interesting case of application of the model to inform policy making regarded China. More in particular, a number of agencies and NGOs working in China cooperated with the MI in an attempt to address a wide variety of issues related to achieving more sustainable growth, dealing with resource constraints (e.g. water, agriculture, energy) in the face of a large and growing population, reducing GHG emissions, and promoting more innovation to move down a greener path. The effects of the growth in population, the economy, transportation, energy consumption, food imports on growth prospects have been examined. The study identified also some important cross sector factors, such as that slowing the growth of the per capita size of housing units would have many beneficial effects, including less cement and steel production, leading to less GHG emissions, less

25 http://www.CROSSOVER-‐project.eu/Workshop/WorkshopProgram.aspx

Legend:

� (+) Positive link

� (-) Negative link

Society Economy

Environment

population

health

employment

emissions

-

renewableenergy

+

+

fuel prices

+

water stress

water demand

agricultureproduction

GDP

-

+ +

-

governmentexpenditure

+

+

productivity+

++

fossil fuelconsumption

+

+

-

+

income per capita

+

+

+nutrition

+

+

B

R

Nutrition loop

Agriculture-water loop

B

Health-GDP-energy loop

R

Health-GDPloop

+

+

Causal Diagram Example


48 | P a g e

energy demand, less reduction in arable land for agriculture. These illustrate important cross sector effects and show how policies need to take a broad view of their results. By generating scenarios over the longer term till 2030, the model was able to show how continuing business as usual would lead to significant challenges and tipping points on water demand, agricultural production, and GHG emissions. Then it was demonstrated how policies that would shift level of consumption and innovation would have significant impacts on sustainability, including that some slower increase in overall consumption may be critical for achieving sustainability on these indicators, despite lower GDP growth and job creation. It shows that it is important to develop policies that mitigate the weaker performance while assuring sustainability and that provide everyone with an acceptable living standard. It clearly illustrates the policy challenges faced and lays the basis for developing more effective policies.

Source: “ Consumption and Sustainability: A Quantitative Approach Based on T21 China”, presented by Weishuang Qu at the Transatlantic Research on Policy Modelling Workshop 26

Available Tools

Research about collaborative software has been conducted since the mid 1980's, when computer-‐human interaction, office automation, and support for group work became the focus of research projects. The term computer-‐supported cooperative work (CSCW) was first used in 1984 and focused on the support of small groups of people. Other terms are used as synonyms for CSCW, especially: collaborative computing, computer mediated communication, and group decision support systems. CSCW is defined as a “computer-‐assisted coordinated activity such as communication and problem

26 http://www.CROSSOVER-‐project.eu/Workshop/WorkshopProgram.aspx

Scenario summary for 2030 Low Consump High Consump

Unit Baseline High Tech Low Tech

real GDP RMB2000/Yr 6.67E+13 5.64E+13 7.23E+13

per capita real GDP RMB2000/Yr 46,829 39,745 47,580

unemployment rate % of workforce 6.12% 18.67% 0.00%

total electricity demand

Bn KWH/Yr 8,191 7,124 8,949

total petroleum demand

MT/Yr 1,059 791 1,294

fossil fuel CO2 emission

Ton/Yr 1.16E+10 8.87E+09 1.40E+10

Agriculture Land Ha 1.15E+08 1.19E+08 1.08E+08

total water demand Ton/Yr 6.07E+11 4.61E+11 7.93E+11


49 | P a g e

solving carried out by a group of collaborating individuals" or as a system, which “looks at how groups work and seeks to discover how technology (especially computers) can help them work". The term groupware also stems from the 1980's and is defined as “computer-‐based systems that support groups of people engaged in a common task (or goal) and that provide an interface to a shared environment". Interestingly, some authors see groupware as advanced software that has to provide awareness support, while other authors also understand code management or emailing as groupware systems. In contrast to groupware, CSCW does not only comprise technological aspects of collaboration, but also incorporates psychological, social, and organizational effects.

Collaborative technologies, especially in the field of groupware and CSCW, are typically classified using the time-‐space taxonomy which distinguishes between communication that occurs at the same space or concurrently at different spaces, and communication that occurs in the same time (synchronously) or in different times (asynchronously). This view was established in 1988 by R. Johansen (“GroupWare: Computer Support for Business Teams”, The Free Press, New York) and taken on in various related publications. The following figure depicts the typical time-‐space matrix as presented in these publications.

Figure 8: the Time-‐Space Matrix

The matrix divides collaborative technologies into four possible constellations, while each of these constellations can be supported better or worse by different communication media.

By the way the architecture of a collaborative modelling tool, i.e., a system that supports a group in developing models, is still under investigation. Some authors have suggested groupware systems that help teams in collective sense-‐making which is an important part of the modelling process. Conklin, Selvin, Buckingham and Sierhuis in “Facilitated Hypertext for Collective Sensemaking: 15 Years on from gIBIS”, a paper presented in 2003 during the 8th International Working Conference on the Language-‐Action Perspective on Communication Modelling (Tilburg, The Netherlands), reports on an approach, Compendium, that is the result of 15 years of experience. Compendium combines three different areas: meeting facilitation, graphical hypertext and conceptual frameworks. To make them work, facilitation is viewed as essential to remove the cognitive overhead for the group


50 | P a g e

members. As groupware systems address the important issue of collective sense-‐making they can be used as the core of a collaborative modelling tool. So far these systems are typically tailored for specific modelling languages though. For a collaborative modelling tool they need to be more modular so that any modelling language can be “plugged in” (e.g., other enterprise or information systems modelling languages). In addition, there is also the need for a negotiation component that facilitates structured arguments and decisions regarding modelling choices. Based on this reflections and issues, recently two tools are emerging:

• The COllaborative Modelling Architecture, COMA27, allows group modelling. Any group member can work on the models whenever it suits them. Any participant can contribute in the way they can: by just looking at proposals and commenting them, by making minor changes to them or maybe even by making their own proposals. The facilitator can see the status of the modelling process at any time and can decide whether a certain proposal should be adopted or needs improvement based on the comments by the other group members and his own judgment.

Figure 9: COMA, COllaborative Modelling Architecture

COMA's design has been inspired by theoretical insights from organizational semiotics and driven by observations of group modelling behavior. The tool is implemented in Visual C++ 2005 on Windows based on the UML Pad and with the wxWidgets GUI library28.

• The OCOPOMO eParticipation platforms, deployed by Open Collaboration for Policy Modelling FP7 project, that will end in December 2012.

27 www.coma.nu 28 http://www.wxwidgets.org/


51 | P a g e

Figure 10: OCOPOMO eParticipation Platform

The platform is a suite of ICT tools for:

o Iterative development of policies in a form of narrative scenarios;

o Policy modelling, creation of agent-‐based formal policy models;

o Open and transparent collaboration in the process of policy development;

o Seamless, goal-‐oriented information exchange between all the stakeholders (policy analysts, operators, decision makers, wider interest groups, general public, etc.);

o Simulation and visualisation of policy alternatives and their consequences;

A First prototype was released in autumn 2011 and tested on a 1st round of pilot applications started on winter 2011. 2nd pilot applications and evaluation started in autumn 2012, and the platform has been released in December 2012.


This research challenge is connected to the research on Web 2.0 and the next generation web. As far as the Policy Modelling in Governance is concerned, this research challenge bridges the gap between citizens and decision makers. It permits an early stage evaluation of the decision maker mental models by opening a dialogue with citizens and allows for an exchange of perspectives. It finally enables the collaboration in the public policy modelling process with the use of a rigorous and formal scientific process.

Current research

According to current research, the following issues are being explored:

• Group model building and systems thinking, focusing on models when tackling a mix of interrelated strategic problems to enhance team learning, foster consensus, and create commitment; although people have different views of the situation and define problems differently, this current field of research shows that this can be very productive if and when people learn from each other in order to build a shared perspective.


52 | P a g e

• Web 2.0 tools for collaboration, as recently pointed out in the FP7 project OCOPOMO (Open Collaboration in Policy Modelling), which aim to implement collaborative scenario building and policy modelling via an integrated ICT toolbox. OCOPOMO provides an innovative "off the mainstream" bottom-‐up approach to policy development, combined with advanced ICT tools and techniques supporting open collaboration. The project is developing an ICT-‐based environment integrating lessons and practical techniques from complexity science, agent based social simulation, foresight scenario analysis and stakeholder participation in order to formulate and monitor social policies to be adopted at several levels. The project is co-‐funded by the European Commission under the 7th Framework Program, Theme 7.3 (ICT for Governance and Policy Modelling).

Future research

Future research should therefore focus on:

• Collaborative Internet-‐based modelling tools, allowing more than one modeller to cooperate, at the same time, on a single model.

• Definition of frameworks allowing even “low-‐skilled” citizens to provide their contribution (even if in a discursive way) to the modelling process.

• Design of more intuitive and accessible Human-‐Computer Interfaces.


53 | P a g e

3.1.3. Easy Access to Information and Knowledge Creation


According to a cybernetic view of intelligent organisations knowledge supersedes 1. the facts, 2. data (statements about facts) and 3. meaningful information (what changes us), the last also defined as “the difference that makes the difference”. Knowledge most often defined as “whatever is known, the body of truth, information and principles acquired” by a subject on a certain topic. Therefore knowledge is always embodied in someone. It implies insight, which, in turn, enables orientation, and thus may be also use as a potential for action (when we are able to use information in a certain environment, then we start to learn, which is the process that helps developing and grounding knowledge). Two more concepts come after knowledge on the same scale, and are Understanding and Wisdom. Understanding is the ability to transform knowledge into effective action, i.e. in-‐depth knowledge, involving both deep insights into patterns of relationships that generate the behaviour of a system and the possibility to convey knowledge to others, whereby wisdom is a higher quality of knowledge and understanding the ethical and aesthetic dimensions.

The research challenge is related to the elicitation of information which, in turn, during the overall model building and use processes will help decision makers to learn how a certain system works and ultimately to gain insights (knowledge) and understanding (apply the extracted knowledge from those processes) in order to successfully implement a desired policy. It is important to note that other research fields (in particular, ICT disciplines) tend to misuse the word “knowledge” and invert it with ”information”.


Proper information acquisition and knowledge development are the key aspect in all research fields, so this research challenge has a horizontal importance for research in general. According to the general need for policy assessment and evaluation, there are some specific issues stemming from this research challenge, which are strongly related to governance:

• Public data use and thus public information elicitation (by citizens)

• Citizens’ behavioural data which are gradually becoming essential for any policy assessment process

• Interoperability of public IT systems

• Creation of a common understanding on a certain system’s behaviour (by means of learning) in order to develop a shared vision on the problems that a certain policy might want to overcome


In current practice, information is drawn from data stored in different types of media (mainly DBMS/ERPs). Web 2.0 has further transformed the way we create data and elicit information from data. Data availability ceased to pose problems as a result of:

• The Internet growth and its uptake

• User Generated Content in Social Networks


54 | P a g e

• Cooperation of IT systems from different organisations thanks to the Service-‐Oriented Architectures (even among old legacy systems), which resulted also in private data availability

• Public Administration Transparency and Public Data use/reuse

Available Tools

A review of the available tools is to be finalized.


The knowledge is still mostly created and passed on by formal methods of teaching, even though the advents of the e-‐Learning, m-‐Learning and webinar fields allow for an increased possibility to perform Distance Learning on the Web. But, since knowledge is developed and grounded by the learning process through action in the environment, the learning in real life comes from committing mistakes. In the field of real life governance, it entails implementing a wrong policy and observing the positive and negative consequences that this policy generates (for example due to a system’s “policy resistance”). Learning of successes is also important, as the A.I. method is based on positive psychology. At present, thanks to the increasing data availability, information elicitation process is much easier, either by tacitly bringing users (data generators) to provide data in a guided way (according to a pre-‐set framework for data input) or with a help of a specific process (e.g.: consultations in e-‐Participation tools).

Current research

According to current research, the main focus is put on the Knowledge Management field or also (more properly, as in our case) to the Knowledge Elicitation field. The latter basically encompasses the following steps:

• Data retrieval and extraction

• Data analysis and interpretation (which usually produces information)

• Data/information adaptation and integration (this is particularly the case where information needs to be used in a model)

Future research

There is still a large field to be explored – the methods of extraction of meaningful information from unstructured sources of data, e.g. when analysing free texts, which applies to all sources of User-‐Generated Content (forums, wikis, social networks, etc.), where the semantic dimension is essential to derive meaningful information rather than just quantitatively analysing the syntax of text. In general, a lot of data is generated by citizens and particularly by their behaviour online, so that the available aggregated data sets contains information on what a citizen does, what s/he likes, how s/he behaves in certain environments, and so on. This data is considered very valuable both for private and public organisations (even though under privacy restrictions which have to be properly addressed).

Also, according to the knowledge creation and development of understanding (regarding a specific system), there is some research currently carried out on how to improve the learning process via the use of e-‐Learning systems. In this respect, it is crucial to boost the research on micro-‐worlds, i.e.


55 | P a g e

complex virtual environments where reality is somehow reproduced and where a decision maker is trained in order to implement his/her strategies and hypothesis and perform what-‐if analysis without the need to necessarily learn from mistakes in real life.

Future research will thus have to focus on the following issues:

• Information elicitation by analysing and interpreting data, also taking into account the semantic point of view.

• Creation of proper micro-‐worlds (or ILEs, Interactive Learning Environments), where the acquired information on a certain system is used (by means of actions), and knowledge is developed by observation of the outcomes of the actions. Also, ILEs will have to be integrated into LMS (Learning Management Systems) in order to extend the potential of distance learning practices, eventually also in a cooperative way (mass learning).

• Interoperability of data sources in order to integrate/aggregate different types of data and be able to automatically infer information from more meaningful datasets.

• In view of the “Internet of Things”, the provision of “portable” models/tools for citizens in order to gather valuable data based on citizens’’ real behaviours. Moreover, these models and tools would enable citizens to check the results of their actions by analysing in real-‐time the response of the model to the information they are contributing to generate, and thus evaluating the eventual benefits they are receiving from their virtuous behaviour or harm they are creating either to their environment or to themselves (e-‐Cognocracy).


56 | P a g e

3.1.4. Model Validation


Policy makers need and use information stemming from simulations in order to develop more effective policies. As citizens, public administration and other stakeholders are affected by decisions based on these models, the reliability of applied models is crucial. Model validation can be defined as ”substantiation that a computerised model within its domain of applicability possesses a satisfactory range of accuracy consistent with the intended application of the model” (Schlesinger, 1979). Therefore, a policy model should be developed for a specific purpose (or context) and its validity determined with respect to that purpose (or context)29. If the purpose of such a model is to answer a variety of questions, the validity of the model needs to be determined with respect to each question. A model is considered valid for a set of experimental conditions if the model’s accuracy is within its acceptable range, which is the amount of accuracy required for the model’s intended purpose. The substantiation that a model is valid is generally considered to be a process and is usually part of the (total) policy model development process (Sargent, 2008). For this purpose, specific and integrated techniques and ICT tools are required to be developed for policy modelling.

Model validation is composed of two main phases:

• Conceptual model validation, i.e. determining that theories and assumptions underlying the conceptual model are correct and that the model’s representation of the problem entity and the model’s structure, logic, and mathematical and causal relationships are “reasonable” for the intended purpose of the model.

• Computerised model verification ensures that computer programming and implementation of the conceptual model are correct, as well as states that the overall behaviour of the model is in line with the available historical data.


Model Validation is connected both to modelling and simulation. According to the general need for policy assessment and evaluation, there are some specific issues stemming from the Model Validation, which are strongly related to governance:

• Reliability of models: policy makers use simulation results to develop effective policies that have an important impact on citizens, public administration and other stakeholders. Model validation is fundamental to guarantee that the output (simulation results) for policy makers is reliable.

• Acceleration of policy modelling process: policy models must be developed in a timely manner and at minimum cost in order to efficiently and effectively support policy makers. Model validation is both cost and time consuming and should be automated and accelerated.

29 Some researchers claim that the category "validity" has little meaning in relation to policy models, as they are generally a form of narrative or storytelling so that their value comes in the act of obtaining a better understanding of the system and being able to communicate concepts effectively and spur discussion between different stakeholders.


57 | P a g e

• Modular and re-‐usable models: a policy model developer deciding to re-‐use existing models or compose them, stumble across the issue of models’ reliability. Model validation can be used for certifying this reliability and creating a database of validated models.


In current practice the most frequently used is a decision of the development team based on the results of the various tests and evaluations conducted as part of the model development process. Another approach is to engage users in the validation process. When developing large-‐scale simulation models, the validation of a model can be carried by an independent third-‐party. Needless to say, that the third party needs to have a thorough understanding of the intended purpose of the simulation model. Finally, the scoring model can be used for testing the model’s validity (e.g. see Balci 1989; Gass 1983; Gass & Joel 1987). Scores (or weights) are determined subjectively when conducting various aspects of the validation process and then combined to determine category scores and an overall score for the simulation model. A simulation model is considered valid if its overall and category scores are greater than some passing score.

Available Tools



Typically all above-‐mentioned approaches are applied after the simulation model has been developed. Performing a complete validation effort after the simulation model has been finalised requires both time and money. However, conducting model validation concurrently with the development of the simulation model enables the model development team to receive inputs earlier on each stage of model development. Therefore, ICT tools for speeding up, automating and integrating model validation process into policy model development process are necessary to guarantee the validity of models with an effective use of resources.

Current research

In Current research, there are a large number of subjective and objective validation techniques used for verifying and validating the modules and the overall model. Robert G. Sargent at the Syracuse University in 2010 provided a relevant ones: Animation; Comparison to Other Model; Degenerate Tests; Event Validity; Extreme Condition Tests; Face Validity; Historical Data Validation; Historical Methods; Internal Validity; Multistage Validation; Operational Graphics; Parameter Variability / Sensitivity Analysis; Predictive Validation; Traces; and Turing Tests. Furthermore, he described a new statistical procedure for validating simulation and analytic stochastic models using hypothesis testing when the amount of model accuracy is specified. This procedure provides for the model to be accepted if the difference between the system and the model outputs are within the specified ranges of accuracy. The system must be observable to allow data to be collected for validation.

Future research

Future research should explore the following issues:

• In order to speed up and reduce the cost of a model validation process, user-‐friendly and collaborative statistical software should be developed, possibly combined with expert systems and artificial intelligence.


58 | P a g e

• Due to the big gap between theory and practice, the considerable opportunity exists for the study and application of rigorous verification and validation techniques. In the current practice, the comparison of the model and system performance measures is typically carried out in an informal manner.

• Complicated simulation models are usually either not validated at all or are only subjectively validated; for example, animated output is eyeballed for a short while. Therefore, complexity issues in model validation may be better addressed through the development of more suitable methodologies and tools.

• Model validation is not a discrete step in the simulation process. It needs to be applied continuously from the formulation of the problem to the implementation of the study findings as a completely validated and verified model does not exist. Validation and verification process of a model is never completed.

• As the model developers are inevitably biased and may be concentrated on positive features of the given model, the third party approach (board of experts) seems to be a better solution in model validation.

• Considering the ranges that simulation studies cover (from small models to very large-‐scale simulation models), further research is needed to determine with respect to the size and type of simulation study

o Which model validation approach should be used,

o How should model validation be managed,

o What type of support system software for model validation is needed.

• Validating large-‐scale simulations that combine different simulation (sub-‐) models and use different types of computer hardware such as in currently being done in HLA (Higher Level Architecture). A number of these VV&A issues need research, e.g. how does one verify that the simulation clocks and event (message) times (timestamps) have the same representation (floating point, word size, etc.) and validate that events having time ties are handled properly.


59 | P a g e

3.1.5. Immersive Simulation


As policy models grow in size and complexity, the process of analysing and visualising the resulting large amounts of data becomes an increasingly difficult task. Traditionally, data analysis and visualisation were performed as post-‐processing steps after a simulation had been completed. As simulations increased in size, this task became increasingly difficult, often requiring significant computation, high-‐performance machines, high capacity storage, and high bandwidth networks. Computational steering is an emerging technology that addresses this problem by “closing the loop” and providing a mechanism for integrating modelling, simulation, data analysis and visualisation. This integration allows a researcher to interactively control simulations and perform data analysis while avoiding many of the pitfalls associated with the traditional batch / post processing cycle. This research challenge refers to the issue of the integration of visualisation techniques within an integrated simulation environment. This integration plays a crucial role in making the policy modelling process more extensive and, at the same time, comprehensible. In fact, the real aim of interactive simulation is, on the one hand, to allow model developers to easily manage complex models and their integration with data (e.g. real-‐time data or qualitative data integration) and, on the other hand, to allow the other stakeholders not only to better understand the simulation results, but also to understand the model and, eventually, to be involved in the modelling process. Interactive simulation can dramatically increase the efficiency and effectiveness of the modelling and simulation process, allowing the inclusion and automation of some phases (e.g. output and feedback analysis) that were not managed in a structured way up to this point.


Immersive simulation is a particular aspect of simulation. As far as the Policy Assessment in Governance is concerned, this challenge may:

• Accelerate the simulation process: policy makers would be able to analyse simulation results, eventually run new scenarios and make decisions as soon as possible and at the minimum cost.

• Collaborative environment: the bigger is the number of stakeholders involved in policy modelling and simulation process, the greater is the necessity of an interactive simulation environment that allows non-‐experts to use the model and understand results as well as permit experts to easily understand new requirements and consequent modification.

• Citizen engagement: interactive simulation tools help to engage citizens in policy-‐making process and to display to them in a simple way the results.

• Data integration: interactive simulation tools allow better managing of a large number and different types of data and information, both for input and output/feedback analysis.


In current practice, data analysis and visualisation, albeit critical for the process, are often performed as a post-‐processing step after batch jobs are run. For this reason, the errors in validating


60 | P a g e

the results of the entire simulation may be discovered only during post-‐processing. What is more, the decoupling of simulation and analysis/visualisation can present serious scientific obstacles to the researcher in interpreting the answers to “what if” questions. Given the limitations of the batch / post processing cycle, it might be advisable to break the cycle and improve the integration of simulation and visualisation. Implementation of an interactive simulation and visualisation environment requires a successful integration of the many aspects of scientific computing, including performance analysis, geometric modelling, numerical analysis, and scientific visualisation. These requirements need to be effectively coordinated within an efficient computing environment. Recently, several tools and environments for computational steering have been developed. They range from tools that modify performance characteristics of running applications, either by automated means or by user interaction, to tools that modify the underlying computational application, thereby allowing application steering of the computational process.

Available Tools



The development of immersive tools is still based on model developers needs and therefore a gap still exists between requirements of policy makers and those of developers. In a collaborative modelling environment, interaction is fundamental in order to speed up the process and make ICT tools user-‐friendly for all the stakeholders involved in the policy model development process.

Current research

In the current research, interactive visualisation typically combines two main approaches: providing efficient algorithms for the presentation of data and providing efficient access to the data. The first advance is evident albeit challenging. Even though computers continually get faster, data sizes are growing at an even more rapid rate. Therefore, the total time from data to picture is not decreasing for many of the problem domains. Alternative algorithms, such as ray tracing (Nakayama, 2002) and view dependent algorithms (Lessig, 2009) can restore a degree of interactivity for very large datasets. Each of those algorithms has its trade-‐offs and is suitable for a different scenario. The second advance is less evident but very powerful. Through the integration of visualisation tools with simulation codes, a scientist can achieve a new degree of interactivity through the direct visualisation and even manipulation of the data. The scientist does not necessarily wait for the computation to finish before interacting with the data, but can interact with a running simulation. While conceptually simple, this approach poses numerous technical challenges.

Future research

With regard to future research, interactive simulation plays a crucial role in a collaborative modelling environment. The trade-‐off between the possibility of enlarging models and including several kinds of data, and the number of people that can understand and modify the model should be deeply analysed. For this purpose, some fundamental issues must be approached:

• Systems should be modular and easy to extend within the existing codes.

• Users of the systems should be able to add new capabilities easily without being experts in systems programming.


61 | P a g e

• Input / output systems should be easily integrated.

• Steering systems should be adaptable to hardware ranging from the largest of supercomputing systems to low-‐end workstations and PCs.

3.1.6. Output Analysis and Knowledge Synthesis


Inputs driving a simulation are often random variables, and because of this randomness in the components driving simulations, the output from a simulation is also random, so statistical techniques must be used to analyse the results. In particular, the output processes are often non-‐stationary and auto-‐correlated and classical statistical techniques based on independent identically distributed observations are not directly applicable. In addition, by observing a simulation output, it is possible to infer the general structure of a system, so ultimately gaining insights on that system and being able to synthesise knowledge on it. There is also the possibility to review the initial assumptions by observing the outcome and by comparing it to the expected response of a system, i.e. performing a modelling feedback on the initial model. Finally, one of the most important uses of simulation output analysis is the comparison of competing systems or alternative system configurations.

Visualisation tools are essentials for the correct execution of this iterative step. The present research challenge deals with the issue of output analysis of a policy model and, at the same time, of feedback analysis in order to incrementally increase and synthesise the knowledge of the system.


Output analysis is a specific aspect of simulation. According to the general need for policy assessment and evaluation, there are some specific issues stemming from the output analysis, which are strongly related to governance:

• Acceleration of policy assessment process: automated output analysis tools would help policy makers to efficiently and effectively analyse the impacts of a policy even if the large number of simulation data must be taken into account

• Citizen engagement: user-‐friendly automated tools for output analysis can be offered to citizens in order to share the simulation results and better engage them in policy-‐making process.


In the current practice a large amount of time and financial resources are spent on model development and programming, but little effort is allocated to analyse the simulation output data in an appropriate manner. As a matter of fact, a very common way of operating is to make a single simulation of somewhat arbitrary length run and then treat the resulting simulation estimates as being the "true" characteristics of the model. Since random samples from probability distributions are typically used to drive a simulation model through time, these estimates are realisations of random variables that may have large variances. As a result, these estimates could, in a particular simulation run, differ greatly from the corresponding true answers for the model. The net effect is that there may be a significant probability of making erroneous inferences about the system under study. Historically, there are several reasons why output data analysis was not conducted in an appropriate manner. First, users often have the unfortunate impression that simulation is just an


62 | P a g e

exercise in computer programming. Consequently, many simulation studies begun with heuristic model building and computer coding, and end with a single run of the program to produce "the answers." In fact, however, a simulation is a computer-‐based statistical sampling experiment. Thus, if the results of a simulation study are to have any meaning, appropriate statistical techniques must be used to design and analyse the simulation experiments and ICT tools must be developed to make the process more effective and efficient. In addition, there are some important issues of output analysis that are not strictly connected to statistics. In particular, an evident gap in literature regards the analysis and integration of feedbacks in modelling and simulation process. Actually, stakeholders are involved, in a post-‐processing phase, in order to analysis the results (more often only the elaboration of them) and understand something about the policy. Sometimes they are able to give a feedback on the difference between their expectations and the result but the process is not structured and effective tools are lacking. The development of tools for analysing and integrating feedbacks should be explored in order to enlarge the number of stakeholders involved and, at the same time, to allow efficient and effective modification at each phase of the process, incrementally increasing the knowledge of the model and, consequently, of the given policy.

Available Tools



A fundamental issue for statistical analysis is that the output processes of virtually all simulations are non-‐stationary (the distributions of the successive observations change over time) and auto correlated (the observations in the process are correlated with each other). Thus, classical statistical techniques based on independent identically distributed observations are not directly applicable. At present, there are still several output-‐analysis problems for which there is no commonly accepted solution, and the solutions that are available are often too complicated to apply. Another impediment to obtaining accurate estimates of a model's true parameters or characteristics is the cost of the computer time needed to collect the necessary amount of simulation output data. Indeed, there are situations where an appropriate statistical procedure is available, but the cost of collecting the amount of data dictated by the procedure is prohibitive.

Current research

In current research, main references are Law (1983), Nakayama (2002), Alexopoulos & Kim (2002), Goldsman & Tokol (2000), Kelton (1997), Alexopoulos & Seila (1998), Goldsman & Nelson (1998), Law (2006).

For output analysis, there are two types of simulations:

• Finite-‐horizon simulations. In this case, the simulation starts in a specific moment and runs until a terminating event occurs. The output process is not expected to achieve steady-‐state behaviour and any parameter estimated from the output will be transient in a sense that its value will depend upon the initial conditions (e.g. a simulation of a vehicle storage and distribution facility in a week time).

• Steady-‐state simulations. The purpose of a steady-‐state simulation is the study of the long-‐run behaviour of the system of interest. A performance measure of a system is called a


63 | P a g e

steady-‐state parameter if it is a characteristic of the equilibrium distribution of an output stochastic process (e.g. simulation of a continuously operating communication system where the objective is the computation of the mean delay of a data packet).

Future research

Referring to previous cited works and in particular to Goldsman (2010), future research should further explore following issues:

• ICT tools for supporting or automating output/feedback analysis

• Allowing an incremental understanding of the model (knowledge synthesis)

• Adapting Design Of Experiment (DOE) for policy model simulation

• Use and integration of more-‐sophisticated variance estimators

• Better ranking and selection techniques.


64 | P a g e

3.2. Data-‐powered Collaborative Governance 3.2.1. Big Data

Summary Overview

Current free tools Top market tools Current and Future Research

The freely available tools permit to overcome data limitations, simplify the analytical process and visualize results. The functionalities provided by these software are: -‐Massively parallel processing (MPP) database product for large-‐scale analytics and next-‐gen data warehousing -‐Data-‐parallel implementations of statistical and machine learning methods -‐Visual data mining modelling

-‐Data storage platforms and other information infrastructure solutions

-‐Massive parallel processing (MPP)

-‐Dataflow engines, software interconnect technologies

-‐Data discovery and exploration tools

-‐Built-‐in text analytics, enterprise-‐grade security and administrative tools

-‐Real-‐time analytics processing (RTAP)

-‐Visualization features supporting exploratory and discovery analytics

-‐On-‐line analytical processing (OLAP)

-‐Business intelligence (BI), Data Warehouse (DW)

-‐Enterprise Data Warehouse (EDW)

-‐Technologies for collecting cleaning, storing and managing data: data warehouse; pivotal transformation; ETL; I/O; efficient archiving, storing, indexing, retrieving, and recovery; streaming, filtering, compressed sensing sufficient statistics; automatic data annotation; Large Database Management Systems; storage architectures; data validity, integrity, consistency, uncertainty management; languages, tools, methodologies and programming environments

-‐Technologies for summarizing data and extracting some meaning: reports; dashboard; statistical analysis and inference; Bayesian techniques; information extraction from unstructured, multimodal data; scalable and interactive data visualization; extraction and integration of knowledge from massive, complex, multi-‐modal, or dynamic data; data mining; scalable machine learning; data-‐driven high fidelity simulations; scalable machine learning; predictive modelling, hypothesis generation and automated discovery -‐Technologies for using data a decision tool: Decision Trees, Pro-‐Con Analysis, Rule Based Systems, Neural Networks, Tradeoff based Decisions


Big Data refers to dataset that cannot be stored, captured, managed and analysed by the mean of conventional database software. Thereby Big Data is a subjective rather than a technical definition, because it does not involve a quantitative threshold (e.g. in terms of terabytes), but instead a moving technological one. Keeping that in mind, the definition of Big Data in many sectors ranges from a few terabytes30 to multiple petabytes31. The definition of Big Data does not merely involve the use of very large data sets, but concerns also a computational turn in thought and research (Burkholder, L, ed. 1992).

30 1 terabyte is equal to 1 trillion bytes 31 1 petabyte is equal to 1000 terabytes


65 | P a g e

As stated by Latour (2009) when the tool is changed, also the entire social theory going with it is different. In this view Big Data has emerged a system of knowledge that is already changing the objects of knowledge itself, as it has the capability to inform how we conceive human networks and community. Big Data creates a radical shift in how we think research itself. As argued by Lazer et al. (2009), not only we are offered the possibility to collect and analyze data at an unprecedented depth and scale, but also there is a change in the processes of research, the constitution of knowledge, the engagement with information and the nature and the categorization of reality. The potential stemming from the availability of a massive amount of data is exemplified by Google. It is widely believed that the success of the Mountain View company is due to its brilliant algorithms, e.g. PageRank. In reality the main novelties introduced in 1998, which brought to second generation search engines, involved the recognition that hyperlinks were an important measure of popularity and the use of the text of hyperlinks (anchortext) in the web index, giving it a weight close to the page title. This is because first generation search engines used only the text of the web pages, while Google added two data set (hyperlinks and anchortext), so that even a less than perfect algorithm exploiting this additional data would obtain roughly the same results as PageRank. Another example is the Google’s AdWords keyword auction model. Overture had previously shown that ranking advertisers for a given keyword based purely on their bids was an efficient mechanism. Google improved the tool by adding the data on the clickthrough rate (CTR) on each advertiser's ad, so that advertisers were ranked by their bid and their CTR.


Big Data have a huge impact also in governance and policy making. In fact their benefits apply to a wide variety of subjects:

• Health care: making care more preventive and personalized by relying on a home-‐based continuous monitoring, thereby reducing hospitalization costs while increasing quality. Detection of infectious disease outbreaks and epidemic development

• Education: by collecting all the data on students’ performance, it would be possible to design more effective approaches. The collection of these data is made possible thanks to massive Web deployment of educational activities

• Urban planning: huge high fidelity geographical datasets describing people and places are generated from administrative systems, cell phone networks, or other similar sources.

• Intelligent transportation based on the analysis and visualization of road network data, so as to implement congestion pricing systems and reduce traffic

• The use of ubiquitous data collection through sensors networks in order to improve environmental modelling

• Analysis and clarification of the energy pattern use through data analytics and smart meters, which can be useful for the adoption of energy saving policies avoiding blackouts

• Integrated analysis of contracts in order to find relations and dependencies among financial institutions in order to assess the financial systemic risk

• The analysis of conversation in social media and networks, as well as the analysis of financial transaction carried out by alleged terrorists, which can be used for homeland security

• Assessment of computer security by the mean of the logged information analysis, i.e. Security Information and Event Management

• Better track of food and pharmaceutical production and distribution chain


66 | P a g e

• Collect data on water and sewer usage in order to reduce water consumption by detecting leaks

• Use of sensors, GPS, cameras and communication systems for crisis detection, management and response

• Use of sensors’ data for carbon footprint management

Policy Applications of Big Data Tools

There is a growing body of evidence highlighting the applications of Big Data not only in traditional hard science and business, but also in policy making due to the predictive power of the data. Let us see some applications:

• Predictability of human behaviour and social events. A research team from Northwestern University32 was able to predict people’s location based on mobile phone information generated from past movements. Moreover Pentland from MIT33 conducted a research showing that mobile phones can be used as sensors for predicting human behaviour, as they can quantify human movements in order to explain changes in commuting patterns given for example by unemployment. Recently another research team from Northeastern University was able to predict the voting outcome in the scope of a famous US television programme (American Idol) based on Twitter activity during the time span defined by the TV show airing and the voting period following it34

• Public health. Online data can be used for syndromic surveillance, also called infodemiology35. As an example Google Flu Trends is a tool based on the prevalence of Google queries for flu-‐like symptoms. As shown by Ginsberg et al. (2008)36 it is then possible to use search queries to detect influenza epidemics in areas with a large population of web search users. In fact according to the US Center for Disease Control and Prevention (CDC)37 a great availability of data coming from online queries can help to detect epidemic outbursts before laboratory analysis. Another related tool is the Google Dengue Trend. In this view the analysis of health related Tweets in US by Paul and Dredze (2011)38 found a high correlation between the modeled and the actual flu rate. In the same way Twitter’s data can be analyzed to study the geographic spread of a virus or disease39. Finally we can talk about Healthmap 40 in which data from online news, eyewitness reports, expert-‐curated discussions, official reports, are used to get a thorough view of the current global state of infectious diseases which is visualised on a map

• Global food security. The Food and Agriculture Organization of the UN (FAO) is chartered with ensuring that the world’s knowledge of food and agriculture is available to those who need it when they need it and in a form which they can access and use41. In fact human population will approach 9 billion by 2050, thereby it will be necessary to put in place

32 http://online.wsj.com/article/SB10001424052748704547604576263261679848814.html 33 http://www.nytimes.com/2011/04/24/business/24unboxed.html?_r=1&src=tptw> 34 http://www.mobs-‐lab.org/uploads/6/7/8/7/6787877/american_idol_finale.pdf 35 http://yi.com/home/EysenbachGunther/publications/2006/eysenbach2006c-‐infodemiologyamia-‐proc.pdf 36 http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/p apers/detecting-‐influenza-‐epidemics.pdf > 37 http://www.cdc.gov/ehrmeaningfuluse/Syndromic.html 38 http://www.cs.jhu.edu/%7Empaul/files/2011.icwsm.twitter_health.pdf 39 http://www.ncbi.nlm.nih.gov/pubmed/21573238 40 http://healthmap.org/en/ 41 http://data.fao.org/ and http://www.grdi2020.eu/Repository/FileScaricati/050e1e8a-‐3e69-‐4ba0-‐86a5-‐b8f7c8322ebe.pdf


67 | P a g e

policies aimed at ensuring a sufficient and fair distribution of resources. In fact the world food production will have to increase by 60% by increasing the agricultural production and fighting water scarcity. The online data portal to be launched by FAO will enhance planners’ and decision makers’ capacity to estimate agricultural production potentials and variability under different climate and resources scenarios

• Environmental analysis. In the last United Nations conference on climate (i.e. COP 17) taking place in 2011, The European Environment Agency, the geospatial software company Esri and Microsoft presented the network Eye on Earth42, which can be used to create an online site and group of services for scientists, researchers and policy makers in order to share and analyze environmental and geospatial data. Other three projects launched by these institutions at COP 17 include WaterWatch (using EEA’s water data); AirWatch, (about EEA’s air quality data); and finally NoiseWatch, which is a combination between environmental data with user-‐generated information provided by citizens. Moreover during 2010 United Nations climate meeting (COP 16) Google launched its own satellite and mapping service Google Earth Engine43, which is a combination of a computing platform, an open API and satellite imagery along 25 years. All these tools will be available to scientists, researchers and governmental agencies for analyzing the environmental conditions in order to make sustainability decisions. In this way the government of Mexico created a map of the country’s forest incorporating 53,000 Landsat images, which can be used by the federal authority and the NGOs to make decisions about land use and sustainable agriculture.

• Crisis management and anticipation. In occasion of the Haiti earthquake44: an European Commission’s Joint Research Center team used the damage reports mapped on the Ushahidi-‐Haiti platform45 to show that this crowdsourced data can help predict the spatial distribution of structural damage in Port-‐au-‐Prince. Their model based on 1645 SMS reports crowdsourced data almost perfectly predicts the structural damage of most affected areas reported in the World Bank-‐UNOSAT-‐JRC damage assessment performed by 600 experts from 23 countries in 66 days based on high resolution aerial imagery of structural damage. As for future developments, some researches46 highlight the fact that Big Data can be used for crisis management and anticipation by building up crisis observatories, i.e. laboratories devoted to the collecting and processing of enormous volumes of data on both natural systems and human techno-‐socio-‐economic systems, so as to gain early warnings of impending events. With those capacity would be possible to set up Crisis and Observatories for financial and economic, for armed conflicts, for crime and corruption, for social crisis, for health risks and disease spreading, for environmental changes.

• Global Development. An inspiring example is given by Global Pulse47, which is a Big Data based innovation programme fostered by the UN Secretary-‐General and aimed at harnessing today's new world of digital data and real-‐time analytics in order foster international development, protect the world's most vulnerable populations, and strengthen resilience to global shocks. The programme is rooted on three main pillars: research on new data indicators providing real-‐time understanding of community’s welfare as well as real-‐time feedback on policies; creation of a toolkit of free open-‐source software for mining real-‐time data useful for shared evidence-‐based decisions; the establishment of country-‐level

42 http://www.eyeonearth.org/ 43 http://earthengine.google.org/#intro 44 http://publications.jrc.ec.europa.eu/repository/handle/111111111/15684 45 http://haiti.ushahidi.com/ 46 http://arxiv.org/pdf/1012.0178v5.pdf 47 http://www.unglobalpulse.org/about-‐new


68 | P a g e

innovation centres (Pulse Lab) where real-‐time data are applied to development challenges. The programme encompasses 5 main projects carried out with several partners:

o “Daily Tracking of Commodity Prices: the e-‐Bread Index”48, which investigates how scraping online prices could provide real-‐time insights on price dynamics

o “Unemployment through the Lens of Social Media” 49 , which relates the unemployment statistics with unemployment-‐related conversation from open social web

o “Twitter and the Perception of Crisis Related Stress”50, which investigates what indicators can help in understanding people’s concerns on food, fuel, finance, housing

o “Monitoring Food Security Issues through New Media”51, which finds emerging trends related to food security using text analysis, semantic clustering and networks theory

o “Global Snapshot of Wellbeing – Mobile Survey”52, aimed at experimenting new tools able of replicating the standards of traditional household surveys in real-‐time on a global scale

• Intelligence and security. As examples of governments’ commitment to Big Data for national security we can present the Cyber-‐Insider Threat (CINDER) 53 program, which aims at developing new ways for detecting cyber espionage activities in military computer networks as well as at increasing the accuracy, rate and speed with which cyber threats are detected. Another example is the Anomaly Detection at Multiple Scales (ADAMS)54 program led by the Defense Advanced Research Project Agency (DARPA), which addresses the problem of anomaly-‐detection and characterization in massive data sets. The program will be initially applied to insider-‐threat detection, in which individual actions are recognized as anomalous with comparison to a background of routine network activity. Finally the Center of Excellence on Visualization and Data Analytics (CVADA) of the Department for Homeland Security (DHS) is leading a research effort on data that can be used by first responders to tackle with natural disasters and terrorists attacks, by law enforcement to border security concerns, or to detect explosives and cyber threats.

An Interesting Application: Smart Cities

A Smart City is a public administration or authorities delivering services and infrastructure based on ICT which are easy to use, efficient, responsive, open and sustainable for the environment. We can identify six main dimensions55:

• Smart economy, characterized by high standard of living and competitive elements: innovative and entrepreneurship, high productivity, flexibility of labour market, internationalism, ability to transform;

• Smart mobility, i.e. efficient public transportation system, local and international accessibility, availability of ICT-‐infrastructure, sustainability and safety;

48 http://www.unglobalpulse.org/projects/comparing-‐global-‐prices-‐local-‐products-‐real-‐time-‐e-‐pricing-‐bread 49 http://www.unglobalpulse.org/projects/can-‐social-‐media-‐mining-‐add-‐depth-‐unemployment-‐statistics 50 http://www.unglobalpulse.org/projects/twitter-‐and-‐perceptions-‐crisis-‐related-‐stress 51 http://www.unglobalpulse.org/projects/news-‐awareness-‐and-‐emergent-‐information-‐monitoring-‐system-‐food-‐security 52 http://www.unglobalpulse.org/projects/global-‐snapshot-‐wellbeing-‐mobile-‐survey 53 http://www.darpa.mil/Our_Work/I2O/Programs/Cyber-‐Insider_Threat_%28CINDER%29.aspx 54 http://www.darpa.mil/Our_Work/I2O/Programs/Anomaly_Detection_at_Multiple_Scales_%28ADAMS%29.aspx 55 See also the project EuropeanSmartCities at http://www.smart-‐cities.eu/model.html


69 | P a g e

• Smart environment (sustainability of natural resources): low pollution, protection of environment, natural attractiveness;

• Smart people, given by high level of human and intellectual capital, high level of qualification, lifelong learning, social and ethnic diversity, flexibility, creativity;

• Smart living (high quality of life); presence of cultural facilities, healthy environment conditions, individual safety, housing quality, education facilities, touristic attractiveness and social cohesion;

• Smart governance given by citizens’ participation in decision-‐making, the presence of public and social services and of transparent and open governance.

The combination of all the benefits stemming from Big Data in governance, make it evident that the integration of heterogeneous data from various domains holds high potential to provide insights on cities. New technologies will unlock massive amounts of data about all the aspects of the city as well as its citizens. For instance new systems involving energy use at fixed locations (point sources, like house and office) are being implemented by the mean of smart metering as well as the integration of various information systems used to record pricing and activity. Another possibility is given by the extraction of positional and frequency data from social media such as Twitter, Facebook, Flickr and Foursquare. All this data will be used for fulfilling the Smart Cities targets. Let us take into account for instance the transportation system, where diagnosing and anticipating abnormal events such as traffic congestions requires integration of various data like traffic data, weather data, road conditions, or traffic light strategy. Another possibility will be given by e-‐inclusion technologies and open data for governance. One important example of the development of the Smart City concept at large scale is the New York City project “Roadmap for a Digital Future”56, which outlines a path to build on New York City's successes and establish it as the world's top-‐ranked Digital City, based on indices of internet access, open government, citizen engagement, and digital industry growth.

Recent Trends

Big Data is a fast growing phenomenon: as the Google CEO Eric Schmidt pointed out in 2010, currently in two days is created in the world as much information as it was from the appearance of man till 2003. Nowadays57 it is possible to store all the world’s music in a $600 worth disk drive, while Facebook content shared every month amounts to $30 billion. According to the forecast global data will grow at a 40% rate next year while the total IT spending will grow just by 5%. In 2010 users and companies stored more than 13 exabytes of new data, which is over 50,000 times the data in the Library of Congress.

Big Data is also a potential booster for the economy, bearing a $300 billion potential annual value to US health care as well as a $600 billion potential annual consumers surplus from using personal location data globally and a 250 billion Euro potential annual value to European public administration. In fact the European Commission is expected to adopt an Open Data Strategy, i.e. a set of measures aimed at increasing government transparency and creating a €32 billion a year market for public data. Finally as reported last year by the McKinsey Global Institute58, the United States will need 140,000 to 190,000 more workers with deep analytical expertise and 1.5 million more data-‐literate managers. Always according to the McKinsey Global Institute the potential value of global personal location data is estimated to be $700 billion to end users, and it can result in an up to 50% decrease in product development and assembly costs. What’s the growth engine of big data?

56 http://www.nyc.gov/html/mome/digital/html/roadmap/theroadmap.shtml 57 See McKinsey Global Institute (2011) “Big data: The next frontier for innovation, competition, and productivity” 58 http://www.mckinsey.com/Features/Big_Data


70 | P a g e

From one side more “old world” data is produced through “open governance” and digitization. From the other side “new world” data are created are continuously collected in domains such as “in silico” medicine, “in silico engineering” and Internet science. Brand new fields of science are being created: computational chemistry, biology, economics, engineering, mechanics, neuroscience, geophysics, etc. etc. This is true also in humanities, such as the birth of computational social science, based on mobile phones and social network digital traces. A wide array of actors including humanities and social science academics, marketers, governmental organizations, educational institutions, and motivated individuals, are now engaged in producing, sharing, interacting with, and organizing data. All these developments are allowed by the rise of new technologies for data collections: web logs; RFID; sensor networks; social networks; social data (due to the Social data revolution), Internet text and documents; Internet search indexing; call detail records; astronomy, atmospheric science, genomics, biogeochemical, biological; military surveillance; medical records; photography archives; video archives; large-‐scale eCommerce.

Inspiring cases

• The Ion ProtonTM Sequencer59 is a rapid genome-‐scale benchtop sequencer. The tool

allows to perform data analysis in the same day on a single stand-‐alone server.

• The NIH Human Connectome Project60 aims at mapping the neural pathways that

underlie human brain function in order to acquire and share data about the structural

and functional connectivity of the human brain.

• The Models of Infectious Disease Agent Study61 is a collaboration of research and

informatics groups to develop computational models of the interactions between

infectious agents and their hosts, disease spread, prediction systems and response

strategies.

• MyTransport.sg 62 is a portal, developed by the Land Transport Authority (LTA) of

Singapore, providing information and eServices for all land transport users.

• UN Global Pulse63, an innovation initiative launched by the United Nations Secretary-‐

General aimed at exploring how digital data sources and real-‐time analytics

technologies can help policymakers to better protect populations from shocks.

Tools on the market

Freely available tools

There are not many cases of freely available tools for Big Data analysis on the market.

The presence of freely available tools on the market bear many benefits, such as:

• Developers and analysts will use them to experiment with emerging types of data structure so as to develop new and different analytical procedures, he added

59http://www.lifetechnologies.com/global/en/home/about-‐us/news-‐gallery/press-‐releases/2012/life-‐techologies-‐itroduces-‐the-‐bechtop-‐io-‐proto.html.html 60 http://neuroscienceblueprint.nih.gov/connectome/index.htm 61 http://www.nigms.nih.gov/Research/FeaturedPrograms/MIDAS/ 62 http://www.mytransport.sg/content/mytransport/home.html 63 http://www.unglobalpulse.org/about-‐new


71 | P a g e

• Developers and IT professionals contribute their findings and know-‐how back into the industry to drive knowledge exchange

The freely available tools permit to overcome data limitations, simplify the analytical process and visualize results. The functionalities provided by these software are:

• Massively parallel processing (MPP) database product for large-‐scale analytics and next-‐gen data warehousing

• Data-‐parallel implementations of statistical and machine learning methods

• Visual data mining modelling

In this view are very important the free Big Data tools developed by Greenplum for data scientists and developers: MADlib and Alpine In-‐Database Miner64 and Greenplum HD Community Edition65.

Some other software partially for free with important Big Data applications: KNIME66, Weka / Pentaho67, Rapid-‐I RapidAnalytics68, Rapid-‐I RapidMiner69. Finally there is R70, which although was not built for Big Data, it has interesting application in this realm.

Enterprise-‐level software

The enterprise-‐level software is adopted for the following functionalities:

• Open source software based on Apache Hadoop

• Data storage platforms and other information infrastructure solutions

• Shared-‐nothing massively parallel processing (MPP) database architectures

• Dataflow engines, software interconnect technologies

• Data discovery and exploration tools

• Built-‐in text analytics, enterprise-‐grade security and administrative tools

• Real-‐time analytic processing (RTAP) platforms

• Software-‐as-‐a-‐service (SaaS)

• Visualization features supporting exploratory and discovery analytics

• On-‐line analytical processing (OLAP)

• BI/DW (business intelligence and data warehousing)

• EDW (enterprise data warehousing). Examples of these software include: Tableau BI platform71; SAS Data Integration Studio72; SAS High Performance Analytics73; SAS On Demand74; SAND

64 http://www.greenplum.com/community/downloads/analytics-‐tools/ 65 http://www.greenplum.com/community/downloads/database-‐ce/ 66 http://www.knime.org/ 67 http://weka.pentaho.com/ 68 http://rapid-‐i.com/content/view/182/196/ 69 http://rapid-‐i.com/content/view/181/196/ 70 http://www.r-‐project.org/ 71 http://www.tableausoftware.com/products/server 72 http://support.sas.com/documentation/onlinedoc/etls/ 73 http://www.sas.com/software/high-‐performance-‐analytics/in-‐memory-‐analytics/analytics.html 74 http://www.sas.com/solutions/ondemand/


72 | P a g e

Analytic Platform75; SAP BEx76; SAP NetWeaver77; SAP In-‐Memory Appliance (SAP HANA)78; ParAccel Analytic Database (PADB) 79 ; IBM Netezza 80 ; IBM InfoSphere BigInsights 81 ; IBM InfoSphere Streams82; Kognitio WX283; Kognitio Pablo84; EMC Greenplum Database85; Greenplum HD86; EMC Greenplum Data Computing Appliance87; Greenplum Chorus88; Cloudera Enterprise89, StatSoft Statistica90 .

Some other software which have not been built specifically for Big Data applications, but nonetheless can be used for Big Data analytics are: Mathematica91, MatLab92 and Stata93.


In order to enjoy all the potential stemming from Big Data it would be necessary to remove the technological barrier preventing the exchange of data, information and knowledge between, disciplines, as well as to integrate activities which are based on different ontological foundations. Even though Big Data have provided a lot of benefits, many challenges are still to be coped with. For instance Gartner (2011)94 argues that the challenges are not only given by the volume of data, but also by the variety (heterogeneity of data types and representation, semantic interpretation) and velocity (rate of data arrival and action timing). According to the recent research those advancements include95

• Data modelling challenges: data models coherent to the data representation needs; data models able to describe discipline specific aspects; data models for representation and query of data provenance and contextual information; data models and query languages representing and managing data uncertainty, and representing and querying data quality information

• Data management challenges: provide quality, cost-‐effective, reliable preservation and access to the data; protect property rights, privacy and security of sensible data; ensure data search and discovery across a wide variety of sources; connect data sets from different domains in order to create open linked data space data can be unstructured or semi-‐structured with no context; different data format; different data labels used for same data elements; different data entry conventions and vocabularies used; -‐ data entry errors; data

75 http://www.sand.com/analytics/architecture/ 76 http://scn.sap.com/community/business-‐explorer 77 http://www.sap.com/platform/netweaver/index.epx 78http://www.sap.com/solutions/technology/in-‐memory-‐computing-‐platform/hana/overview/index.epx 79 http://www.paraccel.com/ 80 http://www-‐01.ibm.com/software/data/netezza/ 81 http://www-‐01.ibm.com/software/data/infosphere/biginsights/ 82 http://www-‐01.ibm.com/software/data/infosphere/streams/ 83 http://www.kognitio.com/analyticalplatform 84 http://www.kognitio.com/pablo 85 http://www.greenplum.com/products/greenplum-‐database 86 http://www.greenplum.com/products/greenplum-‐hd 87 http://www.greenplum.com/products/greenplum-‐dca 88 http://www.greenplum.com/products/chorus 89 http://www.cloudera.com/products-‐services/enterprise/ 90 http://www.statsoft.com/ 91 http://www.wolfram.com/mathematica/ 92 http://www.mathworks.com.au/products/matlab/ 93 http://www.stata.com/ 94 http://my.gartner.com/portal/server.pt?open=512&objID=202&mode=2&PageID=5553&resId=1727219 2011. Available at http://www.gartner.com/it/page.jsp?id=1731916 95 http://www.grdi2020.eu/Repository/FileScaricati/6bdc07fb-‐b21d-‐4b90-‐81d4-‐d909fdb96b87.pdf


73 | P a g e

sets can be so large they cannot be effectively processed by a single machine; data parallelization and task parallelization96.

• Data service/tools challenges: data tools for most scientific disciplines are inadequate to support research in all its phases so that scientists are less productive than what they might be. In fact there is the need of software able to “clean”, analyse and visualize huge amounts of data. Moreover are missing data tools and policies for the ensuring the cross collaboration and fertilization among different disciplines and scientific realms

As for other issues concerning Big Data, Boyd and Crawford (2011) highlight some of them:

• Relationship between automatic search and the definition of knowledge. At the beginning of the 20th century Ford introduced the mass production, automation and assembly line, reshaping not only the way things are produced, but also the general understanding of labor, the human relationship to work, and the society at large. Fordism consisted in breaking down holistic tasks into atomized and independent ones. In the same way Big Data is a new system of knowledge characterized by a computational turn in science leading to a change in the constitution of knowledge, the process of research and the categorization of reality. But as the Fordism had limits (indeed has been overcome by the Just in Time paradigm), also the specialized Big Data tools are not flawless. Big Data, as a new system of knowledge can change the very meaning of learning itself, with all the possibilities and limitations embedded in the systems of knowing

• Big Data may produce misleading claims of objectivity and accuracy. In the science there is a deep cleavage between qualitative and quantitative scientists. Apparently qualitative scientists would be engaged in creating and interpreting stories, while quantitative scientists would be in the business of producing facts. Needless to say, that is not case as all the objectivity claims come from subjects, who make subjective observation and choices. Moreover data analysis is based on a large number of assumptions (see for instance the asymptotic theory in statistics) and on the other hand even though a model may be mathematically or an experiment may be scientifically valid, the final interpretation is subjective. Other examples are the difficulty of integrating in a consistent way different datasets, the arbitrary choices inherent data cleaning and finally the fact that internet databases may well be affected by bias such as frictions and self-‐selection. In this view, by increasing the quantification space, especially in social sciences, Big Data might support objectivity and accuracy claims which are not really grounded on good sense and reality.

• A higher quantity of data does not always mean better data. In all sciences there is a massive amount of literature (interpretation bias, design standardization, sampling mechanism and question bias, statistical significance and diagnostics) aimed at ensuring the consistency of data collection and analysis. Curiously, Big Data scientists sometimes assume a priori quality of their data and completely neglects the methodological issues proper of global sciences. A clear example is given by social media data, which are subject to self-‐

96 Some Big Data challenges are deeply related with policy making, such as the fact that many agencies pay a high premium to both internal resources and external third parties to manage their data. Additionally, data management can sometimes be redundant if not properly set up. Moreover regulations do not take into account the new, expanded capabilities that IT offers as it takes time to issue a new law and bureaucrats are not so keen to novelty


74 | P a g e

selection bias as people using social media is not representative of the society itself. Even the definition of active user and account of a social media might not be innocuous: in fact it is estimated that 40% of Twitter’s users are merely “listeners”, i.e. do not proactively take part. Finally it has to be recognized that in my contexts high quality research is purposely carried out with a limited amount of data, such as for instance in game theory experimental analysis.

• Big Data and Ethical Issues. The use for research purposes of “public” data on social media websites opens the door to deontological issues. The problem is: can those data be used without any ethical of privacy consideration? Obviously Big Data is an emerging field of science, thereby ethical consideration are yet to be fully considered. How the researchers can be sure that their activity is not harmful for some of their subjects? On one hand is impossible to ask for data use permission from all the subjects present in a database. On the other hand, the mere fact that the data are available does not justify their use. Accountability to the field of research and accountability to the research subjects are the ethical keys for Big Data. In all the traditional fields of science, researcher must follow a series of professional standards aimed at protecting the rights and well being of human subjects. On the other hand the ethical implications of Big Data research are not yet clear.

• Digital divides created by Big Data. It is widely accepted that doing research on Big Data automatically involves having a quick and easy access to databases. This is not the case, as only social media companies have access to large datasets, and sell those data at a high price, offering only small data sets to university based researchers. So researchers with a considerable amount of founding or based inside those firms can have access to data that the outsiders will not. Thereby their methodologies and claims cannot be verified. In this view Big Data can create a new digital divide, between researchers belonging to the top universities and working with the top companies, and scholar belonging to the periphery. But the digital divide can be also skills based: in fact only people with a strong computational background are able to wrangle through APIs and analyse massive quantities of data. Concluding there is a new digital divide between the Big Data reach, who are able to analyze and to buy datasets, and belong to top universities and companies, and the Big Data poor, who are outsiders

Finally according to the UN97 the Big Data challenges can be divided along two main dimensions.

The data management:

• Privacy. The development of new technologies always raises privacy concerns for individuals, companies and societies. This is a very crucial issue as privacy, safety and diversity are important for defending the freedom of citizens, and obviously companies have the right to retain their confidential information. In the era of Big Data, the primary producers, who are the citizens using services and devices generating data, are seldom aware that they are doing so or how their data will be used. Sometimes it is also unclear to what extent users of social media such as Twitter consent to the analysis of their data. The pool of individual information shared by mobile phones and credit card companies, social

97 http://unglobalpulse.org/


75 | P a g e

media and research engines is simply astonishing. People must be conscious of that, as privacy is a freedom pillar.

• Access and sharing. A great amount of data is available online for the most disparate uses. On the other hand much data is retained by companies which are concerned about their reputation, the necessity to protect their competitiveness or simply lack the right incentive to do so. On the other hand there is a bunch of technical and regulatory arrangements which has to be put in place in order to ensure inter-‐comparability of data and interoperability of systems.

Data analysis:

• Summarising the data. Sometimes the data might be simply false or fabricated, especially with user-‐generated text-‐based data (blogs, news, social media messages). In addition sometimes data are derived from people’s perceptions, as in calls to health hotlines and online searches for symptoms. Another case is related to opinion mining and sentiment analysis, in which the true significance of the statements can be misled, so that the human factor is always crucial in the analysis. Another problem is that sometimes data are generated from expressed intentions in blogposts, online searches, mobile-‐phone systems for checking market price, which are not a sure indicator of actual intentions and final decisions. So there is a huge problem in summarizing facts from users’ generated text, as there might be a difficulty in distinguishing feeling from facts.

• Interpreting data. A very important concern is given by the sample selection bias, given by the fact that people generating data are not representative of the entire population. For instance younger generations use more internet and mobile devices. In this way the conclusions of the analysis are valid only for the sample at hand and cannot therefore be generalized. Sometimes dealing with huge amounts of data leads the researchers to focus on finding patterns or correlations without concentrating on the underlying dynamics. One thing is to find a correlation, another is to detect a causal relationship. Even more difficult is to identify the direction of the causal relationship without using a founding theory. A final issue is very much linked with using data from different sources, which can magnify the existing flaws in each database

Finally we have the challenges identified by the community white paper drafted with the collaboration of a group of leading researchers across the United States98:

• Heterogeneity and incompleteness. Data must be structured prior to the analysis in an homogeneous way, as algorithms unlike humans are not able to grasp nuance. Most computer systems work better if multiple items are stored in an identical size and structure. But an efficient representation, access and analysis of semi-‐structured data is necessary because as a less structured design is more useful for certain analysis and purposes. Even after cleaning and error correction in the database, some errors and incompleteness will remain, challenging the precision of the analysis.

• Human collaboration. Even if analytical instruments gained tremendous advancements, there are still many realms in which the human factor is able to discover patterns algorithms

98 http://imsc.usc.edu/research/bigdatawhitepaper.pdf


76 | P a g e

cannot. An example can be found in the use of CAPTCHAs, which can discern human users from computer programmes. In this view a Big Data system cannot must involve a human presence. Given the complexity of today’s world, there is the necessity to harness human ingenuity from different domains through crowdsourcing. Thereby a Big Data system requires technologies able to support this kind of collaboration even in case of conflicting statements and judgments.

Current Big Data Techniques

Big datasets can be analysed by the mean of several techniques coming from statistics and computers science.

A list of the principal categories is:

• Cluster analysis. Statistical technique consisting in splitting an heterogeneous group into smaller subsets of similar elements, whose characteristics of similarities are not known in advance. A typical example is to identify consumers with similar patterns of past purchases in order to tailor most accurately a given marketing strategy

• Crowdsourcing. Technique for the collection of data which have been drawn from a large group or community in response to an open call through a networked media such as the internet. This category bears a crucial importance in our case as it is a mass collaboration instance of using Web 2.0

• Data mining. Combination of database management, statistics and machine learning methods useful for extracting patterns from large datasets. Some examples include mining human resources data in order to assess some employee characteristics or consumer bundle analysis to model the behavior of customers

• Machine learning. Subfield of computer science (in the scope of artificial intelligence) regarding the definition and the implementation of algorithms allowing computers to evolve their behaviour based on empirical evidence. An example of machine learning is the natural language processing.

• Natural language processing. Set of computer science and linguistic methods adopting algorithms to analyse natural human language. Basically this field, which began as a branch of artificial intelligence, deals with the interaction between computer and human language

• Neural networks. Computational models which are structured and work similarly to biological neural networks existing among brain cells, and that are used to find in particular non-‐linear patterns in the data. Some applications include game-‐playing and decision making (backgammon, chess, poker) and knowledge discovery in data bases

• Network analysis. Part of graph theory and network science which describes the relationships among discrete nodes in a graph or a network. In particular the social network analysis studies the structure of relationship among social entities. Some applications are include the role of trust in exchange relationships and the study of recruitment into political movements and social organizations

• Predictive modelling. Branch a mathematical model used to best predict the probability of an outcome. This technique is widely used in customer relationship management to produce customer-‐level models able to assess the probability that a customer would take a particular action, such as cross-‐sell, product deep-‐sell and churn


77 | P a g e

• Regression. Statistical method for assessing how the value of a dependent variable changes with one or more dependent variables. Examples of applications include the change in consumer’s behaviour due to manufacturing parameters or economic fundamentals

• Sentiment analysis. Natural language processing methods for extracting information such as polarity, degree and strength of the sentiment over a given feature, aspect of product. Many companies assess how different customers and stakeholders react to their products and action by applying this analysis to blogs, social networks and other social media

• Spatial analysis. Methods for assessing the geographical, geometric or topological characteristics of a data set. The spatial data are often drawn from geographical information systems (GIS) including addresses or latitude/longitude coordinates, to be incorporated into spatial regressions (correlation between commodity price and location) or simulations

• Simulation. Consists in modelling the behavior of a complex system for performing forecast and scenario analysis. As example we can mention Monte Carlo simulations, which are a class of computational algorithms that rely on repeated random sampling to compute their results

Current and Future Research

• Technologies for collecting cleaning, storing and managing data: datawarehouse; pivotal transformation; ETL; I/O; efficient archiving, storing, indexing, retrieving, and recovery; streaming, filtering, compressed sensing sufficient statistics; automatic data annotation; Large Database Management Systems; storage architectures; data validity, integrity, consistency, uncertainty management; languages, tools, methodologies and programming environments

• Technologies for summarizing data and extracting some meaning: reports; dashboard; statistical analysis and inference; Bayesian techniques; information extraction from unstructured, multimodal data; scalable and interactive data visualization; extraction and integration of knowledge from massive, complex, multi-‐modal, or dynamic data; data mining; scalable machine learning; data-‐driven high fidelity simulations; scalable machine learning; predictive modelling, hypothesis generation and automated discovery Technologies for using data a decision tool: Decision Trees, Pro-‐Con Analysis, Rule Based Systems, Neural Networks, Tradeoff based Decisions (which incorporates Reporting, Statistics, Knowledge Based systems)


78 | P a g e

3.2.2. Opinion Mining and Sentiment Analysis

Summary overview

Current free tools Top market tools

Current research Short term future research

Long term future research

-‐ Filtering opinion based on rating; assessing sentiments based on keywords; visual word counting

-‐ Argument mapping

-‐ Machine learning + human analysis

·∙ -‐ -‐ Statistical + Semantic analysis through lexicon/corpus of words with known sentiment for sentiment classification

·∙ -‐ Identification of policy -‐ opinionated material to be analysed

·∙ -‐ Computer-‐generated reference corpuses in political/governance field

·∙ -‐ Visual mapping of bipolar opinion

·∙ -‐ Identification of highly rated experts

·∙ -‐ Visual representation

·∙ -‐ Audiovisual opinion mining

·∙ -‐ Real-‐time opinion mining

·∙ -‐ Machine learning algorithms

·∙ -‐ Natural language interfaces

·∙ -‐ SNA applied to opinion and expertise

·∙ -‐ Bipolar assessment of opinions

·∙ -‐ Multilingual reference corpora

·∙ -‐ Recommendation algorithms

·∙ -‐ Multilingual audiovisual opinion mining

·∙ -‐ Usable, peer-‐to-‐peer opinion mining tools for citizens

·∙ -‐ Non-‐bipolar assessment of opinion

·∙ -‐ Automatic irony detection


The explosion of social media has created unprecedented opportunities for citizens to publicly voice their opinions, but has created serious bottlenecks when it comes to making sense of these opinions. At the same time, the urgency to gain a real-‐time understanding of citizens concerns has grown: because of the viral nature of social media (where attention is very unevenly distributed) some issues rapidly and unpredictably become important through word-‐of-‐mouth.

Policy-‐makers and citizens don’t yet have an effective way to make sense of this mass conversation and interact meaningfully with thousands of others.

As a result of this paradox, the public debate in social media is characterized by short-‐termism and auto-‐referentiality. Many experts consider social media as a missed opportunity for better policy debate.

At the same time, the sheer amount of raw data is also an opportunity to better make sense of opinions. The key asset that Google exploited to reach dominance in the search market is not a better algorithm, but the power of more data.


79 | P a g e

We are therefore at a crucial underpinning where the challenge of information overload can become not a problem, but an opportunity for making sense of a thousand voices and identify problems as soon as they arise.

Opinion mining can be defined as a sub-‐discipline of computational linguistics that focuses on extracting people’s opinion from the web. The recent expansion of the web encourages users to contribute and express themselves via blogs, videos, social networking sites, etc. All these platforms provide a huge amount of valuable information that we are interested to analyse. Given a piece of text, opinion-‐mining systems analyse:

·∙ Which part is opinion expressing;

·∙ Who wrote the opinion;

·∙ What is being commented.

Sentiment analysis, on the other hand, is about determining the subjectivity, polarity (positive or negative) and polarity strength (weakly positive, mildly positive, strongly positive, etc.) of a piece of text – in other words:

·∙ What is the opinion of the writer

Opinion mining and sentiment analysis cover a wide range of applications.

1. Argument mapping software helps organising in a logical way these policy statements, by making explicit the logical links between them. Under the research field of Online Deliberation, tools like Compendium, Debatepedia, Cohere, Debategraph have been developed to give a logical structure to a number of policy statement, and to link arguments with the evidence to back it up99.

2. Voting Advise Applications help voters understanding which political party (or other voters) have closer positions to theirs. For instance, SmartVote.ch asks the voter to declare its degree of agreement with a number of policy statements, then matches its position with the political parties.

3. Automated content analysis helps processing large amount of qualitative data. There are today on the market many tools that combine statistical algorithm with semantics and ontologies, as well as machine learning with human supervision. These solutions are able to identify relevant comments and assign positive or negative connotations to it (the so-‐called sentiment).

The first two point reflect mature application areas, while the third area is emerging and with relevant research issues. We will therefore mainly focus on this area for the research issues.


These applications are the basic infrastructure of large scale collaborative policy-‐making. They help making sense of thousands of interventions. They help to detect early warning system of possible disruption in a timely manner, by detecting early feedback from citizens. Traditionally, ad hoc

99 Other similar tools include Rationale (http://rationale.austhink.com/tour)


80 | P a g e

surveys are used to collect feedback in a structured manner. However, this kind of data collection is expensive, as it deserves an investment in design and data collection; it is difficult, as people are not interested in answering surveys; and ultimately it is not very valuable, as it detects “known problems” through pre-‐defined questions and interviewees, but fails to detect the most important problems, the famous “unknown unknown”. Opinion mining is helpful to identify problems by listening, rather than by asking, thereby ensuring a more accurate reflection of reality.

Argument mapping software is then useful to ensure that policy debates are logical and evidence-‐based, and do not repeat the same arguments again and again.

These tools would finally be helpful not only for policy-‐makers, but also for citizens who could more easily understand the key points of a discussion and participate to the policy-‐making process.

Recent trends

Opinion mining is not in itself a new research theme. Automated methods for content analysis have been increasingly used, and have increased at least 6 folds from 1980 to 2002 (Neuendorf, K. A. 2002. The Content Analysis Guidebook. Sage). The research theme is based in long established computer science disciplines, such as Natural Language Processing, Text Mining, Machine Learning and Artificial Intelligence, Automated Content Analysis, and Voting Advise Applications.

However, according to Pang and Lee (2008), since 2001 we see a growing awareness of the problems and opportunities, and “subsequently there have been literally hundreds of papers published on the subject.”

What is new today is the sheer increase in the quantity of unstructured data, mainly due to the adoption of social media, that are available for machine learning algorithm to be trained on. Social media content by nature reflects opinions and sentiments, while traditional content analysis tended to focus on identifying topics ((Pang, Lee, and Vaithyanathan 2002). As such, it deals with more complex natural language problems. Because of the combination of increase in the volume of data available and more complex concepts to analyse, in recent years there has been a decrease in interest on semantic-‐based application, and a move towards greater use of statistics and visualisation. Just as any other scientific discipline, also automated content analysis is becoming a data-‐intensive science.

Inspiring cases

• Usage of DiscoverText in government100

• OpinionSpace101

• Project NOMAD102

100 http://www.discovertext.com/Government.html 101 http://www.state.gov/opinionspace/ 102 http://www.nomad-‐project.eu/


81 | P a g e

Tools on the market

The market of opinion mining tools is crowded with solution providers. Most of these applications are geared towards analysing customers feedback about products and services, and therefore skewed towards sentiment analysis that detects positive/negative feelings by interpreting natural language.


Most of the state-‐of-‐the-‐art argument mapping and voter advise applications are freely available, because they derive largely from academic community or NGOs. A comprehensive list of such tools is available in http://groups.diigo.com/group/CROSSOVERproject/content/tag/argumentmapping and http://groups.diigo.com/group/CROSSOVERproject/content/tag/VAA

There are currently freely available applications that simply analyse terms based on a pre-‐defined glossary, and giver highly simplified and unreliable results. One example is http://twitrratr.com/

Figure 11: Twitrratr

Another stream of simple, free and popular solutions is the word visualisation. Wordclouds are becoming more and more used to make sense of large quantities of information in a snapshot. Obviously, such tools are also extremely simplified and only offer a visualisation of the most common used terms, which is helpful to have an idea of what the document is about, but little more. Tools such as wordle.com provide an appealing design solution that can serve as an entry level in the opinion mining market. They are therefore important to involve a much wider public in this kind of activities.


82 | P a g e

Figure 12: Wordclouds

Finally, another way of making sense of large amount of information is by relying on human effort, by crowdsourcing and collective intelligence: people are not only submitting their opinions, but actually filtering them by signalling the most important ones. Tools such as uservoice.com allow customers to submit feedback and to rank other people ideas, thereby allowing the emergence of the most popular ideas. These tools are available at very low cost, but research shows that they are effective in gathering feedback but not in identifying good ideas, as voting tends to focus on easier and most popular issues.

Figure 13: UserVoice


83 | P a g e

Enterprise-‐level software

Beside these simple and free applications, there is then a flourishing market of enterprise-‐level software for opinion mining which much more advanced features. These tools are largely in use by companies to monitor their reputation and the feedback about products on social media. In the government context, opinion mining has long been in use as an intelligence tool, to detect hostile or negative communications (Abbasi 2007). More recently, politics has become a key area of applications, as politicians monitor public opinion on social media to understand public reaction to their position.

Technically, these tools rely on machine learning with regard to identifying and classify relevant comments, through a combination of latent semantic analysis, support vector machines, "bag of words" and Semantic Orientation. This process requires significant human effort aided by machines: all the tools on the market rely on a combination of machine and human analysis, typically using machines to augment human capacity to classify, code and label comments.

Automated analysis is based on a combination of semantic and statistical analysis. Recently, because of the sheer increase in the quantity of datasets available, statistical analysis is becoming more important.


Current solutions for opinion mining and sentiment analysis are rapidly evolving, typically by reducing the amount of human effort needed to classify comments.

Among the challenges identified we can select:

-‐ The detection of spam and fake reviews, mainly through the identification of duplicates, the comparison of qualitative with summary reviews, the detection of outliers, and the reputation of the reviewer (Liu 2008)

-‐ The limits of collaborative filtering, which tends to identify most popular concepts and to overlook most innovative / out of the box thinking

-‐ The risk of a filter bubble (Pariser 2011), where automated content analysis combined with behavioural analysis leads to a very effective but ultimately deviating selection of relevant opinions and content, so that the user is not aware of content which is somehow different from his expectations

-‐ The asymmetry in availability of opinion mining software, which can currently be afforded only by organisations and government, but not by citizens. In other words, government have the means today to monitor public opinion in ways that are not available to the average citizens. While content production and publication has democratized, content analysis has not.

-‐ The integration of opinion with behaviour and implicit data, in order to validate and provide further analysis into the data beyond opinion expressed

-‐ The continuous need for better usability and user-‐friendliness of the tools, which are currently usable mainly by data analysts

Current research

Current research is focussing on:

● Improving the accuracy of algorithm for opinion detection

● Reduction of human effort needed to analyze content

● Semantic analysis through lexicon/corpus of words with known sentiment for sentiment classification


84 | P a g e

● Identification of policy opinionated material to be analysed

● Computer-‐generated reference corpuses in political/governance field

● Visual mapping of bipolar opinion

● Identification of highly rated experts

Future research: long term and short term issues

We can distinguish between long and short term research efforts. As for the first one we have:

• Enhanced discoverability of content through Linked Data

• Visual representation

• Audio-‐visual opinion mining

• Real-‐time opinion mining

• Machine learning algorithms

• SNA applied to opinion and expertise

• Bipolar assessment of opinions

• Multilingual reference corpora

• Comment and opinion recommendation algorithm

• Cross-‐platform opinion mining

• Collaborative sharing of annotating/labelling resources

On the other hand, for the long-‐term:

● Autonomous machine learning and artificial intelligence

● Usable, peer-‐to-‐peer opinion mining tools for citizens

● Non-‐bipolar assessment of opinion

● Automatic irony detection


85 | P a g e

3.2.3. Visual Analytics for collaborative governance: the opportunities and the research challenges

Summary Overview

Market availability Challenges and gaps Current research Short term future research

Long term future research

-‐Information visualisation requirements for business intelligence and situational awareness -‐Enterprise knowledge visualisation linking -‐Online analytical processing and data mining -‐Advanced social network analysis and visualisation -‐Data mining and interactive visualisation communication of location-‐based statistical data -‐Information visualisation tools for high dimensional non-‐linear data -‐Visual analysis of data in spreadsheet format

-‐ Demographics visualisations, allowing stakeholders and decision makers to have a clear picture of the data and of their trends over time -‐ Legal Arguments visualisation: text analysis, argumentation mappings and visualisation algorithms -‐ Discussion Arguments visualisation, making use of visualisation techniques for visualizing a discussion’s flow -‐Geographic visualisation tools -‐Financial markets monitoring and visualizing in real time -‐Advanced applications for security and defense

-‐Close the loop of information selection, preparation and visualisation -‐Simultaneous multiple visualisation -‐Integration of visualisation with comments / wiki / blogs -‐Collaborative platform display Interaction between visualisation and models -‐Mobile visual analytics tools -‐Geo-‐visualisation of government data -‐Integration with opinion mining and participatory sensing -‐Evaluation framework for visualisation effectiveness -‐Visualisation infrastructures for policy modelling issues

-‐Re-‐usable, mashable tools for visual analytics -‐Tighter integration between automatic computation and interactive visualisation -‐Bias identification and signalling in visualisation -‐Perceptual, cognitive and graphical principles -‐Efficiency of the visualisation techniques to enable interactive exploration interaction techniques such as focus & context -‐Impact evaluation of visual analytics on policy choices

-‐Learning adaptive algorithm for users intent -‐Advanced visual analytics interfaces -‐Intuitive affordable visual analytics interface for citizens -‐Development of novel interaction algorithms incorporating machine recognition of the actual user intent and appropriate adaptation of main display parameters such as the level of detail, data selection, etc. by which the data is presented


86 | P a g e


The explosion in computing techniques led to the generation of a tremendous amount of data which are stored in the internet and processed in the IT infrastructures all over the world. Some examples of new technologies for data collection are: web logs; RFID; sensor networks; social networks; social data (due to the Social data revolution), Internet text and documents; Internet search indexing; call detail records; astronomy, atmospheric science, genomics, biogeochemical, biological; military surveillance; medical records; photography archives; video archives; large-‐scale eCommerce.

In managing this huge amount of data, when it comes to human-‐computer interaction there is a need to distil the most important information to be presented it in a humanly understandable and comprehensive way. Here it comes visualisation, which is a way to interpret and translate data from computer understandable formats to human ones by employing graphical models, charts, graphs and other images that are conventional for humans (Bederson and Shneiderman 2003). From one hand we can define visualisation as any technique for creating create insight, preferably by allowing users to interact and alter with the visualisation to iteratively solve questions and form new questions based on previous findings. On the other hand visualisation can be defined as a set of techniques for communicating knowledge that can be supported by data.

In contrast with visualisation traditionally seen as the output of the analytical process, visual analytics 103 considers visualisation as a dynamic tool that aims at integrating the outstanding capabilities of humans in terms of visual information exploration and the enormous processing power of computers to form a powerful knowledge discovery environment. In this view visual analytics is useful for tackling the increasing amount of data available, and for using in the best way the information contained in the data itself. Moreover visual analytics aims at present the data in way suitable for informing the policy making process.

More in particular the interdisciplinary field of visual analytics aims at combining human perception and computing power in order to solve the information overload problem. In Thomas and Cooks (2005) definition, visual analytics is “the science of analytical reasoning supported by interactive visual interfaces”. Precisely visual analytics is an iterative process that involves information gathering, data preprocessing, knowledge representation, interaction and decision making. The characteristic of this field is that it entails the association of data-‐mining and text-‐mining technologies, used for preprocessing massive amounts of data, and information visualisation104, which is useful for disentangling important from trivial and useless information. In a certain way information visualisation becomes a tool in a semi-‐automated analytical process characterized by the cooperation between humans and computers, in which is the user who decides the direction of the analysis relating to a particular task, while the system works as an interaction tool. It is somehow difficult to distinguish among information visualisation and visual analytics. In poor terms we can say that information visualisation handles abstract data structures such as trees or graphs, and finally visual analytics deals properly with sense-‐making and reasoning. More in particular information visualisation is mostly applied to data not belonging to scientific inquiry, e.g. graphical representations of data for business, government, news and social media. Visualisation work does

103 http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=1573625 104 We can define Information visualisation as a way of making data easier to understand using direct sensory experience, rather than linguistic or logical reasoning. Or in the words of Friendly, information visualisation is the study of "the visual representation of large-‐scale collections of non-‐numerical information, such as files and lines of code in software systems, library and bibliographic databases, networks of relations on the internet, and so forth". (See Michael Friendly 2008)


87 | P a g e

not necessarily deal with an analysis task nor does it always use advanced data analysis algorithms. On the other hand visual analytics can be seen as an integral approach to decision-‐making, combining visualisation, human factors and data analysis. It entails identifying the best algorithm for a given analysis task, to be integrated with the best automated analysis algorithms with appropriate visualisation and interaction techniques.

Visualisation and visual analytics should be considered in strict integration with other research areas, such as modelling and simulation105, social network analysis, participatory sensing, open linked data, visual computing.

The disciplines in the domain of visualisation and visual analytics include: Human-‐Computer Interaction (HCI), Computer Science, Graphic and Information Design, Usability Engineering, Cognitive and Perceptual Science, Decision Science, Information Visualisation, Scientific Visualisation, Databases, Data Mining, Statistics, Knowledge Discovery, Data Management & Knowledge Representation, Presentation, Production and Dissemination, Statistics, Interaction, Geospatial Analytics, Graphics and Rendering, Cognition, Perception, and Interaction.

As far the visual analytics methodologies are concerned, in the CROSSOVER taxonomy we can identify the following: visualisation of a single, static, embedded data set; visualisation of multiple static data sets; visualisation of a single live data feed or updating data set; and finally visualisation of multiple data points, including live feeds or updates.


Today’s governments face the challenge of understanding an increasingly complex and interdependent world, and the fast pace of change and increased instability in all the areas of regulation requires rapid decision making able to draw on the wider amount of available evidence in real-‐time. How can visualization and visual analytics help?

• Generate high involvement of citizens in policy-‐making. One of the main applications of visualisation is in making sense of large datasets and identifying key variables and causal relationships in a non-‐technical way. Similarly, it enables non-‐technical users to make sense of data and interact with them. For instance, the GapMinder106 software helps to understand the main global demographic changes and raise awareness on the implications of sound health policies in developing countries.

• Understand the impact of policies: visualisation is instrumental in making evaluation of policy impact more effective. For instance, Farmsubsidy107 helps understanding who are the main beneficiaries of the common agricultural policy by geo-‐referencing the single beneficiary.

• Identify problems at an early stage, detect the “unknown unknown” and anticipate crisis: visual analytics are largely used in the intelligence community because they help exploiting the human capacity to detect unexpected patterns and connections between data. Thereby they help early detection of potential threats at an early stage. For

105 The connections between simulation and visualisation appears even more clear when dealing with user interfaces, which enable the visualisation to take user commands 106 http://www.gapminder.org/ 107 http://farmsubsidy.org/


88 | P a g e

instance, the VisAware108 project in the US provides situational awareness in situation of emergencies, helping the coordination of different resources involved in emergencies

History and trends

Since from the beginning of human history, visualisation has been an effective way to communicate both abstract and concrete ideas. The appearance of digital visualisation led to the development of graphic hardware as well as to a wide array of technique used to visualize data in a number of ways (van Wijk, 2005).

One of the best-‐known examples of visualisation dates back to the 19th century with the drawings109 by Charles Joseph Minard, who developed a format to show data tied to a timescale with a landscape background. In particular Minard conveyed a complex series of events through various data measures, explained together with their causes and consequences in a single graphic. Minard's drawings are applied to show the march of Napoleon’s army towards Moscow, starting with 422,000 and ending with 10,000 men, and Hannibal's crossing of the Alps, starting with 97,000 and ending with 6,000 men. The modern visualisation field, making use of computer graphics, originated in the late 1980s with the studies on scientific visualisation applied to fluid dynamics, volume visualisation, molecular modelling, imaging remote-‐sensing data, and medical imaging (Rosenblum 1994). From scientific visualisation took place some more recent areas, such as information visualisation, mobile visualisation, location-‐aware computing and visual analytics. Information visualisation arose when Robertson, Card and Mackinlay in the 1980s started to use the work of Bertin (1967) and Tufte (1983) in interactive computer applications. Later Shneiderman (1996) inter al. formalized the process of information visualisation. Finally Ware (2004) emphasized the important of human perception in information visualisation. In parallel with information visualisation raised the field of data mining, aimed at discovering information hidden in massive amounts of data. A characteristic of the field is that it aimed at substituting the human analysis with automatic computer operations, not supporting human perception with interactive visualisation. In order to avoid that was developed the interdisciplinary field of visual analytics, which combines human perception abilities with computers’ processing power in order to tackle massive amounts of information. Visual analytics can therefore be seen as the combination between human factors and data analysis on one side, and information visualisation (Keim et el. 2008). Future developments of visual analytics include the fields of enhanced collaboration capabilities, more intuitive interaction, support of non-‐computing devices, as well as the integration of quantitative and qualitative data. In fact visual analytics require particular technological advances, as traditional data mining tools are unsuitable for some necessary functionalities such as the algorithm speed required for iterative visualisation.

Inspiring cases in information visualisation and visual analytics:

• GapMinder110

• US Labour Force visualisation111

• State Cancer Profiles112

108 http://www.sci.utah.edu/publications/yarden05/VisAware.pdf 109 http://www.math.yorku.ca/SCS/Gallery/minbib/index.htm. For other examples please refer to http://www.infovis.net/printMag.php?num=110&lang=2 110 http://www.gapminder.org/ 111 http://flare.prefuse.org/launch/apps/job_voyager 112 http://statecancerprofiles.cancer.gov/micromaps/


89 | P a g e

• Instant Atlas113

• Rennes Metropole114

• City Dashboard115

• OECD Better Life Index116

• Gain Index117

• IBM Many Bills118

• Graphical Contingency Analysis119

• DeepCity3D120

• Vis Sense121

Projects in information visualization and visual analytics:

• Jigsaw122: visualization for investigative analysis

• Ploceus123: network-‐based visualization of tabular data

• Dotlink360124: visual analytics for exploring converging business ecosystems

• SportsVis125: visualization to analyze sports data

• Intelligence Analysis126: visual analytics to help intelligence analysts

• SellTrend127: visualizing temporal, categorical event transactions

• Dust & Magnet128: InfoVis via a magnet metaphor

• Fund Explorer129: stock portfolio diversification through Context Treemaps

113 http://www.instantatlas.com/CDC_story.xhtml 114 http://dataviz.rennesmetropole.fr/quisommesnous/en/ 115 http://citydashboard.org/choose.php 116 http://www.oecdbetterlifeindex.org/ 117 http://index.gain.org/


90 | P a g e

• InfoCanvas130: peripheral information art

• Information Mural131: squeezing large data sets into small views

• NetVizor132: visualizing network topologies

• SunBurst133: radial space-‐filling views of hierarchies

• Tarantula134: testing and debugging large software systems

Policy applications of visualisation and visual analytics tools

With regard to the governance and policy making context, some visualisation tools can be applicable to a wide array of issues and situation (education, environment, public health, urban growth, national defense, etc.). In the public context, visual analytics of public data is an exploding field, with particular relation to the open data movement, in order to monitor policy context and evaluate government policies. Most basic mash-‐up tools are available to visualize government.

Let us see some other examples:

• Demographics visualisations, allowing stakeholders and decision makers to have a clear picture of the data and of their trends over time. Visualisation of demographic data make easier the design and evaluation of various policies, as there is no need to dig through acres of numbers. In fact advanced algorithms are able to create figures and illustrations easy to interpret. Typical examples are the aforementioned GapMinder (which embeds visualisations of various demographic data at global level), as well as Dynamic Choropleth Maps135, DataPlace136, Hive Group137, Name Voyager138, State Cancer Profiles139.

118 http://manybills.researchlabs.ibm.com/ 119 http://availabletechnologies.pnnl.gov/technology.asp?id=288 120 http://www.deepcity3d.eu/default.aspx 121 http://www.vis-‐sense.eu/ 122 http://www.cc.gatech.edu/gvu/ii/jigsaw/ 123 http://www.cc.gatech.edu/gvu/ii/ploceus/ 124 http://www.cc.gatech.edu/gvu/ii/dotlink/ 125 http://www.cc.gatech.edu/gvu/ii/sportvis/ 126 http://www.cc.gatech.edu/gvu/ii/intell/ 127 http://www.cc.gatech.edu/gvu/ii/selltrend/ 128 http://www.cc.gatech.edu/gvu/ii/dnm/ 129 http://www.cc.gatech.edu/gvu/ii/fundexplorer/ 130 http://www.cc.gatech.edu/gvu/ii/infoart/ 131 http://www.cc.gatech.edu/gvu/ii/mural/ 132 http://www.cc.gatech.edu/gvu/ii/netviz/ 133 http://www.cc.gatech.edu/gvu/ii/sunburst/ 134 http://pleuma.cc.gatech.edu/aristotle/Tools/tarantula/ 135 http://www.turboperl.com/dcmaps.html 136 http://www.knowledgeplex.org/dataplace.html 137 http://www.hivegroup.com/gallery/worldpop/ 138 http://www.babynamewizard.com/voyager# 139 http://statecancerprofiles.cancer.gov/micromaps/


91 | P a g e

• Legal Arguments visualisation: text analysis, argumentation mappings and visualisation algorithms can be applied to legal documents in order to simplify legislation making it more accessible and comprehensible to the general public (Many Bills140, Clear Congress Project141), or in order to visually represent corroborative evidence (e.g. the tools Carneades142, Deflog143)

• Discussion Arguments visualisation, making use of visualisation techniques for visualizing the flow of a discussion that include various arguments, in order to instantly get awareness of the topics discussed, as well as of the arguments and the support such arguments gain. In this view visualisation supports all interested stakeholders to understand the flow of a discussion, which is presented to them in a structured and interactive format, avoiding numerous discussion threads. Example of such visualisation tools include DebateGraph144, which is intensively used for building argumentation maps, as well as Araucaria145, Compendium146, Argublogging147 and Rationale148.

• Geovisualisation, which is based on the provision of theory, tools and methods for visual analysis, synthesis, exploration and representation of geographical data and information in order to derive problem specific models and design task specific maps for incorporating geographical knowledge into planning and decision making. Some examples of such tools include ESTAT149, GeoViz Toolkit150, the geovisualisation tools at the US National Cancer Institute151, some applications of InstantAtlas152.

• Advanced visualisation applications used for security and national defense. In this fields, software advances are being led both on the military and on the corporate front. In fact business organizations also have urgent information visualisation requirements that support their business intelligence and situational awareness capability, data mining and reporting requirements. In this view many of the software innovations are being targeted at financial and corporate requirements, but are also applicable to the defense domain due to common data mining and information visualisation challenges. Examples of such tools are: DataMontage153, HoneyComb154, Oculus GeoTime155 and Starlight156.

140 http://researcher.watson.ibm.com/researcher/view_project.php?id=1232 141 http://clearcongressproject.com/ 142 http://carneades.berlios.de/downloads/ 143 http://www.ai.rug.nl/~verheij/aaa/ 144 http://www.debategraph.org 145 http://araucaria.computing.dundee.ac.uk/ 146 http://compendium.open.ac.uk/institute/ 147 http://www.arg.dundee.ac.uk/?p=624 148 http://rationale.austhink.com/ 149 http://www.geovista.psu.edu/ESTAT/ 150 http://www.geovista.psu.edu/geoviztoolkit/index.html 151 http://gis.cancer.gov/nci/geovisualisation.html 152 http://www.instantatlas.com/clients.xhtml#government 153 http://www.stottlerhenke.com/datamontage/examples/madcap/Air_force_wargame_simulation.htm 154 http://www.hivegroup.com/solutions/demos/merit.html 155 http://www.oculusinfo.com/papers/GeoTime_Brochure_06.pdf 156 http://starlight.pnl.gov/


92 | P a g e

Other very interesting examples are Analyst’s Notebook 157 is Visual Sentinel Visualizer158, adopted by intelligence agencies such as the CIA.

• Visualisation applications adopted for financial markets monitoring and visualizing in real time. An example of such tool is SmartMoney159.

• Visualisation applied to governmental finances/expenditure monitoring, such as USAspending.gov160, OffenerHaushalt161 and Where Does My Money Go162.

Tools on the market

There is a massive quantity of visualisation tools in the market, both freely available and enterprise level, critical for analysts and researchers, but also for common people, is now available online.


First of all we have visualisation websites useful for sharing and presenting data, provide clear context on important cultural, environmental, social and economic issue, build chart and share visualisation and discoveries. Such examples include Data360163. Moreover there are “do it yourself” infographic tools such as Vizify164, Visual.ly165, Easel.ly166 and Vizualize.me167.

Then we have data visualisation tools used for plotting data on maps, frameworks for creating charts, graphs and diagrams and tools to simplify the handling of data transforming them into spreadsheets, visual data mining and database exploration system, data visualisation system for high-‐dimensional data, visualisation framework for animating data. Some examples of those tools are: Data Wrangler168, JavaScript InfoVis Toolkit169, VisDB170, Graphviz171, IBM OpenDX172, Gephi173, GeoCommons174, Miso Dataset175, Polymaps176, Tableau Public177.

157 http://www.i2group.com/us/products/analysis-‐product-‐line/ibm-‐i2-‐analysts-‐notebook 158 http://www.fmsasg.com/LinkAnalysis/Government/Solutions.asp 159 http://www.smartmoney.com/map-‐of-‐the-‐market/ 160 http://usaspending.gov/ 161 http://bund.offenerhaushalt.de/ 162 http://www.wheredoesmymoneygo.org/ 163 http://www.data360.org/index.aspx 164 https://www.vizify.com/ 165 http://visual.ly/ 166 http://www.easel.ly/ 167 http://vizualize.me/ 168 http://vis.stanford.edu/wrangler/ 169 http://philogb.github.com/jit/ 170 http://bib.dbvis.de/uploadedFiles/202.pdf 171 http://www.graphviz.org/ 172 http://www.opendx.org/ 173 https://gephi.org/ 174 http://geocommons.com/ 175 http://misoproject.com/dataset/ 176 http://polymaps.org/ 177 http://www.tableausoftware.com/public/


93 | P a g e

Tools available in the market

Apart from free visualisation tools, there are also much more advanced software which are used by firms in order to satisfy their information visualisation requirements for business intelligence support and situational awareness capability, as well as data mining and reporting requirements. Other uses include enterprise knowledge visualisation, linking knowledge to spatial data, online analytical processing and data mining, advanced social network analysis and visualisation, data mining and interactive visualisation, communication of location-‐based statistical data, on-‐line and batch environment for business graphics, information visualisation tools for high dimensional non-‐linear data, visual analysis of data in spreadsheet format, analysis of high volumes of unstructured text, analysis of high-‐dimensional data in large complex data sets and of multivariate time-‐oriented data.

Some examples of such software are: CViz Cluster 178 visualisation, IBM ILOG 179 visualisation, Spotfire180, Survey Visualizer181, Infoscope182, Sentinel Visualizer183, Grapheur 2.0184, InstantAtlas185, Miner3D186, VisuMap187, Drillet188, Eaagle189, GraphInsight190, Gsharp191, Tableau192.

Other examples of visualisation software can be found in

• http://groups.diigo.com/group/CROSSOVERproject/content/tag/visualisation

Key Challenges and Gaps

New tools like the Many Eyes Word Tree193, Treemap194, Tag Cloud195 and Bubble Chart196 are available but lack interactivity. What is also missing is a better interaction of visualisation approaches and analytical processes of text mining, as well as a better integration between new opportunities for data collection, such as open data and participatory sensing, policy modelling and

178 http://www.alphaWorks.ibm.com/formula/CViz 179 http://www-‐01.ibm.com/software/websphere/ilog/ 180 http://spotfire.tibco.com/ 181 http://www.macrofocus.com/public/products/surveyvisualizer/ 182 http://www.macrofocus.com/public/products/infoscope/ 183 http://www.fmsasg.com/ 184 http://grapheur.com/ 185 http://www.instantatlas.com/ 186 http://www.miner3d.com/ 187 http://www.visumap.net/ 188 http://drillet.appspot.com/ 189 http://wp.eaagle.com/ 190 http://www.graphinsight.com/ 191 http://www.avs.com/products/gsharp/index.html 192 http://www.tableausoftware.com/ 193 http://www-‐958.ibm.com/software/data/cognos/manyeyes/page/Word_Tree.html 194 http://www.treemap.com/ 195 http://www.tagcloud.com/ 196 See http://manyeyes.alphaworks.ibm.com/manyeyes/, which can be also found in the project CROSSOVER Diigo collection


94 | P a g e

visual analytics tools. Most applications related to visual analytics of public data remain at the level of visualisation only, with limited analytical functionalities.

Visualisation tools are still largely design for analyst and are not accessible to non-‐experts. Intuitive interfaces and devices are needed to interact with data results through clear visualisations and meaningful representations. User acceptability is a challenge in this sense, and clear comparisons with previous systems to assess its adequacy and objective rules of thumbs to facilitate design decisions would be a great contribution to the community.

Scalability of visualisation in face of big data availability is a permanent challenge, since visualisation requires additional performances with respect to traditional analytics in order to allow for real time interaction and reduce latency.

Finally, visualisation is largely a demand-‐ and design-‐driven research area. In this sense one of the main challenge is to ensure the multidisciplinary collaboration of engineering, statistics, computer science and graphic design.

A relevant challenge of visualisation and visual analytics is to adapt existing techniques to policy modelling:

• RelaNet (Landesberger et al. 2008), which displays the network relations and thereby is able to show the connections and co-‐variances of the different opinions overtime

• CirVis3D (Landesberger et al. 2009), which can visualize clustered opinion snippets as well as display time series in order to show the opinion trends over time

Following Chen (2005), who builds on Rhyne et al. (2004), we can enumerate a number of challenges in the topic:

• Usability: the availability of low cost, ready to use and reconfigurable information visualisation systems, as well as a balanced portfolio of general purpose fully functional information visualisation systems is used is crucial

• Understanding elementary perceptual–cognitive tasks: research should not only focus on relatively high level cognitive activities such as browsing and searching, or judging the relevance of information. Rather it should primarily focus on the identification and de-‐codification of visualized objects would be a fundamental step toward engineering information visualisation systems

• Prior knowledge: in order to understand the underlying message in visualized information users need a prior knowledge of how to operate the information visualisation system, as well as the domain knowledge of how to interpret the content

• Education and training: on the one hand there is the need for the need for researchers and practitioners within the field of information visualisation to learn and share principles and skills of visual communication. On the other hand potential users from other fields must realize the value of information visualisation and how it might contribute to their work

• Intrinsic quality measures: finding quality metrics is crucial for the evaluation and selection of visual information advances, and for understanding to what extent an information visualisation design represents the underlying data faithfully and efficiently, and preserves intrinsic properties of the underlying phenomenon

• Scalability: need for the adoption of parallel computing and other high-‐performance computing techniques in information visualisation


95 | P a g e

• Aesthetics: it is important to assess how insights and aesthetics interact in sustaining insightful and visually appealing information visualisation. What visual properties make users think a graph is pretty or visually appealing?

• Paradigm shift from structures to dynamics: shift from the study of the structure of visualisation to the assessment of the dynamic properties of underlying phenomena, providing built-‐in trend detection mechanisms embedded in the data modelling component

• Causality, visual inference, and predictions: there is a strong necessity for the elaboration of sensitive and selective algorithms that can resolve conflicting evidence and suppress background noises. To this respect a great role is played by complex network and link analysis

• Knowledge domain visualisation: it encompasses several of the aforementioned challenges, and it is linked to the fact that it is not only the information conveyed to be important, rather is its structure, which is a social construction

Current research

• Close the loop of information selection, preparation and visualisation

• Multiple, coordinated views in visualisation/visual analytics197

• Integration of visualisation with comments / wiki / blogs

• Collaborative platform display

• Interaction between visualisation and models

• Mobile visual analytics tools, e.g. Sitegeist198

• Geo-‐visualisation of government data

• Integration with opinion mining and participatory sensing

• Evaluation framework for visualisation effectiveness

• Visualisation infrastructures for policy modelling issues

A list of EU funded projects in visual analytics include:

• VisMaster-‐Visual Analytics: Mastering the Information Age199

• VisSense-‐ Visual Analytic Representation of Large Datasets for Enhancing Network Security200

197 See Heer, Jeffrey, Fernanda B. Viégas, and Martin Wattenberg. 2007. Voyagers and Voyeurs: Supporting Asynchronous

Collaborative Information visualisation. In CHI 2007, April 28–May 3, 2007, San Jose, California, USA. See also the presentation on social visualisation carried out by Fernanda Viégas and Martin Wattenberg at the University of Harvard, April 13 2009

198 http://sunlightfoundation.com/blog/2012/12/13/sitegeist-‐uncover-‐the-‐data-‐around-‐you/ 199 http://www.visual-‐analytics.eu/ 200 http://cordis.europa.eu/projects/rcn/94912_en.html


96 | P a g e

• CUBIST-‐ Combining and Uniting Business Intelligence and Semantic Technologies201

• WATTALIST – Modelling and Analysing Demand Response Systems202

• CODE – Commercially empowered Linked Open Data Ecosystems in Research203

• SemSeg-‐4D Space-‐Time Topology for Semantic Flow Segmentation204


Short-‐term research

• Reusability of mashup tools (mashup is a web application which combines data from one or more sources into a single integrated tool or application) for visual analytics

• Tighter integration between automatic computation and interactive visualisation, which consists in the availability of complex and powerful algorithms that allow for manipulating the data under analysis, transforming it in order to feed suitable visualisations

• Bias identification and signalling in visualisation

• Techniques and algorithms for creating effective visualisation tools based on perceptual psychology (dealing with the process by which the physical energy received by sense organs forms the basis of perceptual experience), cognitive science (focusing on how information is represented, processed, and transformed) and graphical principles

• Visualisations enabling interactive exploration techniques such as focus & context, in order for the viewers to be able to see the object of primary interest presented in full detail while at the same time getting a overview–impression of all the surrounding information — or context — available

• Exploiting visualisation as a medium to engage citizens in policy-‐related complex matter

• Visualisation as a way to provide (persuasive) feedback and change in attitudes, opinions, behaviors

• Visualisation as a medium for grassroots/crowd-‐sourced participation, collaboration on data-‐related issues

• Impact evaluation of visual analytics on policy choices

• Research in making visualisation accessible for non-‐experts

Long-‐term research

201 http://cordis.europa.eu/projects/rcn/95904_en.html 202 http://cordis.europa.eu/projects/rcn/100984_en.html 203 http://cordis.europa.eu/projects/rcn/103419_en.html 204 http://www.semseg.eu/


97 | P a g e

• Learning adaptive algorithm for users intent. Note that learning/adaptive algorithms are defined as being capable to automatically change behaviour based on its execution context (data handled by the algorithm, configuration parameters of the runtime environment, resources used) in order to obtain optimal performances

• Advanced visual analytics interfaces: visual interfaces in which neither the analytics nor the visualisation needs to be advanced in itself but synergy between automation and visualisation is in fact advanced

• Intuitive and affordable visual analytics interface for citizens

• Development of novel interaction algorithms incorporating machine recognition of the actual user intent or of the actual relevance for the user and appropriate adaptation of main display parameters such as the level of detail, data selection, etc. by which the data is presented


98 | P a g e

3.2.4. Serious Gaming for Behavioural Change


So far, collaborative ICTs have dramatically augmented the capacity of people to connect and collaborate. Yet, less impact has been achieved in terms of actual change and action, as most collaboration remain confined to an elite of highly-‐motivated individuals and faces the traditional limits of human attention and motivation. As illustrated in other challenges, ICT can improve data collection and analysis, but if attention and motivation are not present, little impact can be achieved. This challenge depicts ICT solutions that enable behavioural change and action. Even when citizens and government are fully aware of necessary policy choices, they might irrationally choose short-‐term benefits.

Simulation and serious gaming (also known as interactive learning environments) offer opportunities to impact on personal incentives to action and showing long-‐term and systemic effects of individual choices, thereby lowering the engagement barrier to collaborative governance and augmenting its impact. In particular, serious games have been developed for educational purposes and raising awareness on particular issues while not requiring high levels of engagement.

Simulation tools enable users to see the systemic and long-‐term impact of their action in a very concrete and tangible form, thereby encouraging more responsible behaviour and long-‐term thinking. Gaming engages users through the “fun” and “social” dimension, thereby providing incentives towards action. Feedback and simulation systems include both individual and government behaviour, thereby allowing policy-‐makers and citizens to detect the impact of both individual and policy choices.

Engagement of domain experts is a crucial issue for building reliable games and simulation tools. Toolkits and modules enable a wider audience of stakeholders to take a direct, active role in games development, thereby enabling all relevant knowledge to be elicited and captured by the simulation and gaming scenarios and models. Pre-‐built toolkit enables the creation directly by thematic experts and not by technology experts.


Most applications of simulation and gaming are developed into the context of education and learning, while more interactive feedback producing systems have been applied to personal health and energy conservation. The specific challenges of gaming for public policy awareness and action are currently less researched, but are very specific because of their large-‐scale interaction and systemic effects of individual behaviour, which characterised this field.

Furthermore, the availability of a simulation toolkit is necessary to empower a diverse and inclusive simulation landscape, where the most diverse set of ideas can be influential and listened to.

Recent trends

Simulation and gaming have started to be applied in different policy contexts in order to engage wider audiences. Games are developed “on purpose”, by highly skilled developers, in the public sector and by civil society, therefore requiring significant investment and without the specific thematic knowledge of the field. Furthermore, existing serious games lack flexibility to allow for unpredictable developments and non-‐linear behaviours, where scenarios evolve and adapt to users choices rather than being rigidly prescribed. Commercial solutions that turn long-‐term effects into short-‐term feedback are available, but still lack usability as well as the fun dimension of games and


99 | P a g e

finally require high levels of engagement. They are designed for individual feedback and do not cover the complexity of systemic interactions, which are typical of public governance issues.

To sum up, serious gaming is still requiring high level of engagement, and progress is needed in terms of usability and appeal in order to reach “casual gamers” including, immersive and emotion aware games.

Current practice

• Purpose-‐built gaming and simulation for understanding of policy issues and of individual behaviour

Public Policy Applications

Simulation and gaming can be useful to policy makers in the following terms (Mayer et al. 2004 , Bots and van Daalen 2007 ):

• Research and analyse a policy issue when it is not feasible to tackle the real system (due to time constraint or just because it does not exists) or to include human behaviour by way of a computer model (due to unrealistic assumptions such as perfect rationality). In this view the game becomes a laboratory which can produce a great deal of data which provide useful insights

• Design alternative solutions to a problem analyse and assess the possible consequences of the alternative solutions in order to recommend a course of action for the policy-‐maker. In this view the game can be seen as a virtual design studio useful to boost out-‐of-‐the-‐box thinking about alternative solutions to a policy issue, and also to ponder recommendations’ consequences

• A game can be used to provide strategic advice acting as a virtual practice ring in which the policy maker can rehearsal different strategies. A typical example of such kind is given by the war games, in which the other players act as sparring partners for the policy makers, playing the role of another stakeholder as opportunistically as possible

• Many policy issues require mediation so that it is necessary to seek for consensus among stakeholders. This can be done by putting the players around a virtual negotiation table by the mean of a mediation game. In this way the changes in attitude and the discovery of new opportunities for conflict resolution are eased by the interaction among stakeholders during the game

• Normally experts and elites are involved in the policy-‐making process, while citizens and ordinary people are completely neglected. However, by defining virtual consultation forums it is possible to allow equal access for all the actors carrying views and opinions, which would have been otherwise disregarded. In this respect using games and simulations bears and advantage given by the fact that ordinary people can focus and express themselves more easily when playing a role

• Clearly ethical questions and opinions have a great influence on the policy making process. Games and simulations can be used to clarify the values and arguments behind a point of view. While in ordinary political debate values remain implicit, by creating a virtual parliament it is possible to make them explicit. Furthermore gaming and simulations can be used to magnify positions and opinions of stakeholders, so that the game can be designed to reward players for quality and clarity of argumentation

Moreover readapting the taxonomy of Sawyer and Smith simulation and serious gaming can be useful in the following domains (cross-‐referenced with game objectives):

• Public sector and NGOs: public health education and mass casualty response (games for health); political games (advergames); employee training (games for training); provide info to the


100 | P a g e

public (games for education); data collection/planning (games for research); strategic and policy planning, spatial planning (games for producing); diplomacy and opinion research (games as work)

• Defence: rehabilitation and wellness (games for health); recruitment and propaganda (advergames); soldier/support training (games for training); school house education (games for education); wargames and planning (games for research); war planning and weapons research (games for producing); command and control (games as work)

• Healthcare: cyber therapy/exergaming (games for health); public health (advergames); policy and social awareness campaigns (games for training); training games for health professionals (games for education); games for patient education and disease management (games for research); visualization and epidemiology; biotech manufacturing and design (games for producing); public health response planning and logistics (games as work)

• Education: inform about diseases/risks (games for health); social issue games (advergames); train teachers/workforce skills (games for training); learning (games for education); computer science and recruitment (games for research); P2P Learning (games for producing); distance learning (games as work)

Inspiring cases

Let us present now some inspiring cases of serious games applied to policy making:

• SimHealth: The National Health Care Simulation is a management simulation of the U.S. Healthcare system released during Congressional debates on the Clinton health care plan

• SimCity 2013: is an upcoming city-‐building/urban planning simulation computer game allowing allows players to visualize data, such as pollution and water distribution, which will be realised in February 2013

• City One: the game teaches industry professionals and civil servants the real-‐world planning in fields such as optimization of banking, retail, energy and water solutions

• Democracy 2: government simulation game in which the player acts as the president or prime minister of a democratic government introducing and altering policies in areas such as tax, economy, welfare, foreign policy, transport, law and order and public services

• Close Combat Marines: serious game for military training purposes, with particular reference to the United States Marine Corp

• Incident Commander™ NIMS-‐compliant training tool for Homeland Security: in this game the player mimics the role of incident commander in case of natural or manmade disaster, terrorist attack or hostage situation. Application: US Department of Justice officers’ training

• Virtual Battlespace Systems 2: this is an interactive military simulator developed for the United States Marine Corp and the Australian Defence Force to meet the individual needs of military, law enforcement, homeland defence, loadmaster, and first responder training environments

• Pulse!! Virtual Clinical Learning Lab for Health Care Training: the game recreates a lifelike, interactive, virtual environment in which civilian and military heath care professionals practice clinical skills in case of catastrophe or terrorist attack


101 | P a g e

• Levee Patroller: immersive 3D game-‐based environment to train levee inspection knowledge and skills, in order to be prepared to cope with unexpected flooding

• Construct.it: game-‐based learning environment allowing players to experience and debrief some of the complexities involved in large-‐scale urban projects. Application: development plan for Scheveningen-‐Harbour of The Hague

• Simport-‐Maasvlakte 2: computer-‐supported multi-‐player simulation game that mimics the real processes involved in planning, equipping and exploiting the new area in the Port of Rotterdam

• Pro Rail: capacity optimization of a complex infrastructural network, in this case the Dutch railways. Applications: cargo capacity management, opening of the Vecht-‐bridge, increase traffic on the A2-‐corridor

• Win Manager: online multiplayer negotiation business game in which players conduct a sequence of bilateral negotiations pursued through private threads on the general game board

• Management Business Game: business game focussed on the simulation of a company’s management in a competitive market, which can be played both online and offline

• Management Utilities Euroshop: management of a chain of retail stores selling electronic products, through which players identify the relationships between management issues and competitive market factors

• Shadow Government: serious game based on the gamification of real countries, systems, and worldwide events. Based on System Dynamics, customized at the country level, it allows players to test several policy interventions and evaluate their impacts


102 | P a g e


Following Mayer (2009) we can identify the following challenges and gaps:

• Cultural changes concerning the interaction between science in politics, democracy. Changes in the role of elites, activism and citizens’ participation, as well as the recent emergence of game cultures

• Changes in public policy making perception, i.e. from rational comprehensive to political and incremental

• How natural and human-‐caused events can influence the political agenda (climate warming, pollution, depletion of natural resources, terrorism)

• Institutional changes and the emergence of new industrial or institutional actors

• Technological innovation in computing and simulation modelling, such as agent-‐based models, cellular automata, or virtual game worlds.

Moreover following IDATE we can display the following key challenges regarding in particular serious gaming:

• Restructure the game in order to cope with specific purposes and broaden the audience

• Innovate the existing business models

• Automating a portion of the production process, such as for example the integration of sector-‐specific elements

• Try to persuade reluctant users and create sector-‐targeting serious gamins and persuading reluctant users

• Investing in all connected platforms

Current research

• Kit-‐based serious games

• Integration between policy models and simulation

• Design of appealing, adaptive and context-‐aware interfaces. Impact of simulation and gaming on individual behaviour

• Unconscious impact of feedback systems

Research disciplines: human-‐computer interaction, sensors, information visualisation, sensor design, psychology, pedagogy, public policy



• Citizens-‐ and experts-‐generated gaming

• Immersive interfaces


103 | P a g e

• Large-‐scale collaboration in development

• Casual serious gaming

• Ethical issues in serious gaming

• User-‐controlled simulation and gaming

• Non-‐linear and adaptive scenarios for gaming in policy context

• Integrated analysis of information and behavioural change

• Impact of simulation and gaming on systemic behaviour


• Augmented reality citizens-‐generated gaming and simulation

• Ubiquitous feedback systems on public governance

• Model and display long-‐term systemic effects of individual choices on public policy topics

• Interplay between different feedback systems and other information outputs

3.2.5. Linked Open Government Data

The notion of Government Data concerns all the information that governmental bodies produce, collect or pay for. This could include geographical data, statistics, meteorological data, data from publicly funded research projects, and digitized books from libraries. In this respect the definition of Open Public Data is applicable when that data can be readily and easily consulted and re-‐used by anyone with access to a computer. In the European Commission's view 'readily accessible' means much more than the mere absence of a restriction of access to the public. Data openness has resulted in some application in the commercial field, but by far the most relevant applications are created in the context of government data repositories. With regard to linked data in particular, most research is being undertaken in other application domains such as medicine. Government starts to play a leading role towards a web of data. However, current research in the field of open linked data for government is limited.

Following the Open Government Working Group Meeting in Sebastopol 205 and the Sunlight Foundation206, there is a set of principles according to which data can be considered open:

• Data must be completed, i.e. no part of them should be omitted due to security, privacy or privilege limitations

• Data must be primary, disaggregated and not modified, and must be published with the finest possible level of granularity

• Data must be timely as their value is time-‐relevant • Data must be accessible to the widest range of users and purposes • Data formats must not under exclusive control of an entity

205 http://opengovdata.org/home/8principles 206 http://sunlightfoundation.com/policy/documents/ten-‐open-‐data-‐principles


104 | P a g e

• Data should not be subject to any copyright • Data must be machine-‐processable • Data access must be non-‐discriminatory • Data must be permanent, so that their embedded information is available over time • Data must be cheaply accessible

In the same way according to Davies et al.207 engaging open data should:

• Be demand driven • Put data in context • Support conversation around data • Build capacity, skills and networks • Collaborate on data as a common resource

Moreover according to Vander Sande et al. 208, publishing data leads to more transparency, new businesses, better evidence-‐based policy making and increased public sector efficiency only if the different actors in the chain have co-‐ownership of the data and be able to participate directly in its correction. In this sense free licensing and shared platform to publish and offer feedback/corrections directly to the data are crucial.

Linked open government data are valuable for a number of reasons. Firstly, openness in government data is important for the economic reasons. For instance, the Open Data Strategy for Europe launched by the European Commission is expected to deliver a €40 billion boost to the EU's economy each year. More in general, open government data are important for participatory decision making:

• Promotion of transparency concerning the destination and use of public expenditure • Improvement in the quality of policy making, which becomes more evidence based • Display the full economic and social impact of information, and create services based

on government data • Increase in the collaboration across government bodies, as well as between

government and citizens • Permits new added-‐values services to come into existence • Increase the awareness of citizens on specific issues, as well as their information about

government policies • Promote accountability of public officials • Very important examples are given within the scope of the Open Government

Initiative 209 carried out by the Obama Administration for promoting government transparency on a global scale:Data.gov210: platform which increases the ability of the public to easily find, download, and use datasets that are generated and held by the Federal Government. In the scope of Data.gov US and India have developed an open source version called the Open Government Platform 211 (OGPL), which can be downloaded and evaluated by any national Government or state or local entity as a path toward making their data open and transparent

• USAspending.gov212: it is a searchable website displaying for each Federal award the name of the entity receiving the award, the amount of the award, information on the award, and the location of the entity receiving the award

207 http://www.w3.org/2012/06/pmod/pmod2012_submission_5.pdf 208 http://www.w3.org/2012/06/pmod/pmod2012_submission_4.pdf 209 http://www.whitehouse.gov/open 210 http://www.data.gov/ 211 http://www.opengovplatform.org/ 212 http://www.usaspending.gov/


105 | P a g e

• FederalRegister.gov213: HTML edition of the Federal Register to make it easier for citizens and communities to understand and get informed about the regulatory process

• Performance.gov214: website providing a window of US Government Administration effort to improve performance and accountability, in order to create a government that is more effective, efficient, innovative, and responsive

• IT Dashboard215: website enabling federal agencies, industry, the general public and other stakeholders to view details of federal information technology investments

At the European level we have the repository of applications making use of open data: publicdata.eu216. At the European national level the initiatives include:

• United Kingdom: Data.gov.uk217, which collects data from 5,400 datasets available, from all central government departments and a number of other public sector bodies and local authorities.

• Italy: Dati.gov.it218, which is an open data portal allowing citizens, developers, firms and public administrations to make use of the public administration information stock

• Spain: datos.gob.es219, the national portal for managing and organizing the Catalogue of Public Information of the General State Administration

• Ireland: StatCentral.ie 220 , providing standard documentation on recurring official statistics and links to where they can be found

• Netherlands: Overheid.nl 221 , the central access point to all information about government organizations of the Netherlands

213 https://www.federalregister.gov/ 214 http://www.performance.gov/ 215 http://www.itdashboard.gov/ 216 http://publicdata.eu/app?page=1 217 http://data.gov.uk/ 218 http://www.dati.gov.it/content/datigovit-‐il-‐portale-‐dei-‐dati-‐aperti-‐della-‐pa 219 http://datos.gob.es/datos/ 220 http://www.statcentral.ie/ 221 http://www.overheid.nl/


106 | P a g e

At transnational level there are the World Bank222, United Nations223, REEP224 and Open Knowledge Foundation225 portals.

As highlighted by the experience of the Open Corporates226, which turn the freely available raw data into something genuinely useful that customers will be prepared to pay for, open data are valuable also for the private sector.

Figure 14 Open Data Business Model (source: Istituto Superiore Mario Boella)

In this case the data can generate revenue in a number of ways:

• Subscriptions or royalties; • The so-‐called “freemium” model where a basic service of offered for free but with

charges for premium services; • Advertising by third parties; • Cross subsidy; • By offering services that are cheaper and more efficient to outsource.

222 http://data.worldbank.org 223 http://data.un.org 224 http://www.reegle.info/ 225 http://opengovernmentdata.org 226 http://www.w3.org/2012/06/pmod/pmod2012_submission_16.pdf


107 | P a g e

Looking at the best practices and examples of Linked Open Government Data, it is possible to refer to diagram maintained by Richard Cyganiak (DERI, NUI Galway) and Anja Jentzsch (HPI), which is a visualization of the key LOD providers and their linkages227:

Figure 15 -‐LOD providers and their linkages

Three other inspiring cases are:

• The clean energy information gateway reegle.info228, which makes use of LOD for providing comprehensive clean energy country profiles so that users can access the highest quality information in a visually appealing fashion. By using reegle.info small organizations can share responsibilities, as they are not required to maintain large databases. Moreover the information is directly linked to data providers, so that updates take place immediately.

• Open Energy Information (OpenEl) 229 , a collaborative knowledge-‐sharing platform providing open and free access to energy related models, tools and data. The business benefits of using this system stem from the fact that a small organization not having a huge team of people for maintaining a large database with information of clean energy, can obtain the same amount of knowledge making use of an overview of a variety of energy-‐related and country-‐specific topics. Moreover, given the direct link to the data providers’ information, any update occurs in real time.

• The UK official government archive Legislation.gov.uk, which offers access to all published UK legislation so that it can be shared by citizens and businesses. The dataset

227 http://lod-‐cloud.net/ 228 http://data.reegle.info 229 ttp://en.openei.org


108 | P a g e

covers more than 800 years and includes the most recent changes in legislation. Legislation.gov.uk merges the contents of the Office of Public Sector Information website and the Statute Law Database in ordered to provide UK public and local acts, church instruments, ministerial orders and acts of the parliament.

• The tool publicspending.gr230, which takes data from a variety of sources and from the combined data the system is able to derive graphs showing where public money is being spent and which departments are spending it. This is the first linked open data application in Greece, where it can used to aid to policy making and transparency

• Another interesting case is “Where Does My Money Go?”231, which shows how daily taxes are allocated among the different functions of the government.

• UK Crime Map232, which has prompted a change in the way police resources are prioritized, and is widely used by the government itself which is benefitting from much more efficient access to information

• The BudgIT233 platform, which turned the Nigerian budget into an interactive document, complete with commentary channels via the Web and SMS

• The DERI's Galway Volvo Ocean Race 234 app for Android and iPhone, created by converting various data sets into linked data and then enrich that data through crowdsourcing. DERI was used to classify 350 apps, showing that the majority of apps: • Have been produced by individuals rather than commercial companies • Are Web based • Combine OGD with maps • Rely on static data sets rather than real time data • Use a single data set, rather than mixed data

• The Open Culture Data project (Open Cultuur Data)235, presenting the results of a hackathon. The winning entry made use of a video dataset and smartphone capabilities to match a person's location with video taken in a given area

• GLAMs236 (galleries, libraries, archives, museums), which provides open access to the cultural heritage

Some other interesting tools are the RDF Data Cube Vocabulary 237 , the Data Cube faceted browser238, Openpolis239, Nosdeputes240, Virtueel Platform241

Current challenges: open data and opinion mining

There is lot of effort for extracting public opinion and sentiment towards policies242. The problem is

230 http://www.w3.org/2012/06/pmod/pmod2012_submission_32.pdf 231 http://www.wheredoesmymoneygo.org/ 232 http://www.police.uk/ 233 http://www.w3.org/2012/06/pmod/pmod2012_submission_8.pdf 234 http://www.w3.org/2012/06/pmod/pmod2012_submission_20.pdf 235 http://www.opencultuurdata.nl/ 236 http://www.w3.org/2012/06/pmod/pmod2012_submission_22.pdf 237 http://www.w3.org/TR/vocab-‐data-‐cube/ 238 http://www.w3.org/2012/06/pmod/pmod2012_submission_12.pdf 239 http://www.openpolis.it/ 240 http://www.nosdeputes.fr/ 241 http://virtueelplatform.nl/nieuws/apps-‐for-‐amsterdam-‐zoekt-‐nieuwe-‐open-‐data-‐apps/ 242 http://www.w3.org/2012/06/pmod/pmod2012_submission_29.pdf


109 | P a g e

that the tools that are better at extracting real data from social media content are of course expensive. In this respect the future challenges for opinion mining applied to social media are:

• How to reduce human efforts; • Identification of good ideas; • Finding necessary investments; • How to improve usability of tools.

We also have to notice that most of social media interaction is not carried out in public: for example in Facebook only open discussion pages can provide information for sentiment analysis and opinion mining.

Current research topics

• Integration of open government data (OGD) and social media data (SMD): policy makers will soon be able to see the subjective reaction to objective changes through a dashboard that is powered by linked data

3.2.6. Collaborative Governance


While all challenges provide opportunities for a more effective large-‐scale collaboration in public action, the relevant institutional design is far from being introduced. The formal inclusion of citizens input in the policy-‐making process, the deriving institutional rules, the legitimacy and accountability framework are all issues that have so far been little explored. Instant, open governance implies a substantial increase in feedback loops that are of a different scale with respect to the present context. Any system stability is affected by the number, speed and intensity of feedback loops, and the institutional context has been designed for less and slower loops.

The definition and design of public sector role is being directly affected by the radical increase in bottom-‐up collaboration, deriving from the lower cost of self-‐organisation. There are also important questions to be answered – where does the legitimacy come from, how to gain and maintain the trust of users, how to identify the users online. There is also a very important issue of how to take into the account the diversity of the standpoints, i.e. how to achieve a consensual answer to controversial social issues, especially when we do not offer alternatives (ready-‐made options) but start from an open question and work throughout different options proposed by participants. Furthermore, the trade-‐off between direct or representative model of democracy will have to be analysed in this context. It is far from being proved that the open and collaborative governance is really inclusive and representative of all the social groups, including the disadvantaged and of all standpoints. There is a visible risk that online collaboration increases the divide, rather than reduces it. However, in the current situation, the circles where the policy proposals are designed, amended and ranked hierarchically are very small and composed by leaders of political parties, top-‐level civil


110 | P a g e

servants, CEOs of large firms. On-‐line collaborative tools would broaden these circles to all those that have a competence in the field being discussed, and have an ability to elaborate an argument.

The management of institutional bodies is changing: innovative ideas and insight coming from employees and citizens are key resources to be exploited, and meritocracy and transparency are entering a once stable and conservative workforce. Enhanced collaboration with citizens and private third parties should be accompanied by adequate legal and accountability frameworks, mapping incentives to participation and enabling business models for different stakeholders.

The privacy paradigm is changing and appropriate, more dynamic frameworks have to be designed, taking into account the willingness of citizens to share information and at the same time ensuring their full awareness of the implications and their control over the data usage.

Recent trends

The current status is characterized by practice-‐driven implementation, accompanied by little scientific reflection. Guidelines and soft regulation are being created from scratch and by building on other institutions examples. The development of collaborative governance is growing rapidly without an appropriate reference framework.

Public Policy Application

As it is widely recognized, policy issues of our age can be addressed only through the collaboration of all the components of the society, including the private sector and individual citizens. In this view the advantages in collaborative governance are given by:

• Effectiveness and efficiency in the delivery of programs

• Professional development / capacity building

• Better needs assessment and use of available resources

• Boost communication among citizens and stakeholders

• Increase transparency and accountability, as well as equity and inclusiveness

• Avoiding duplication in policy making

• Increasing responsiveness, access and build relationship

• Improving public image

• Improve the quality of information

• Consensus based decision-‐making

• Increased acceptance of results

Collaborative governance can be applied to virtually all the policy making fields. The following areas constitute a mere example:


111 | P a g e

• Infrastructure management: building new infrastructures often entails the necessity to balance conflicting interests, especially for what concerns the case of huge externalities

• Digital inclusion: the increase in the use of ICT has to be fostered by the collaboration at every level

• Energy: delivering affordable and efficient energy, collaboration on the definition of energy regulations

• Environment: definition of new regulations on environmental safeguard, mediation concerning the use of environmental resources, collaboration in assessing public projects with environmental impact

• Health care: collaboration in health care reform, awareness and education campaigns, disease prevention

• Transportation: collaboration in the definition of a transportation plan, negotiation of transportation rules, settlement of disputes on the construction of a transportation facility

In general the fields where collaborative governance is most fruitful are, in general, those that (1) involve many different categories of stakeholders, and (2) where the conflicts are not frozen along entrenched lines. Indeed, (1) the benefits of collaboration are enhanced when the diversity is highest, and (2) entrenched conflicts are hardly solved by dialogue, and become a field of power balance and force.

Inspiring cases of ICT applications to Collaborative Governance

The Open Government Initiative 243 carried out by the Obama Administration, for promoting government transparency and citizen engagement on a global scale:

• Partner4Solutions244: the website for the Partnership Fund for Program Integrity Innovation at the Office of Management and Budget. By using this tool, the Partnership Fund aims at gathering ideas from citizens for improving the Federal assistance programme

• Regulations.gov245: in this website it is possible to comment on proposed regulations and related documents published by the U.S. Federal government, as well as to search and review original regulatory documents as well as comments submitted by others

• Challenge.gov246: online challenge platform allowing the public to bring the best ideas and top talent to bear on our nation’s most pressing challenges, which can range from simple ideas and suggestions to proofs of concept, designs, or finished products that solve the grand challenges of the 21st century

• We the People247: allowing citizens to create and launch a petition in order to engage the government

• Change by Us248, which allows people to propose ideas and projects for improving the cities they live. So far the tool has been applied to New York, Phoenix and Philadelphia


112 | P a g e

• Wikirendum249, a platform where citizens can share ideas on policy making

• The Digital Engagement Guide250, which is a platform where ideas on how to use digital and social media in the public sector are shared


The key challenges and gaps in collaborative governance are:

• Coping with accelerating changes in policy making

• Overlapping in institutions and jurisdictions

• Increasing complexity in the issues to be tackled

• Ability to choose the appropriate tool for tackling the problem at hand

• The need to integrate policies and resources

• Managing expectations

• Public involvement processes can be disconnected from real decision making

• Tackle conflicting interests among participants

• Using tools appropriate for the scale (small-‐scale or large scale) of the problem/solution

• Calibrate the level of citizens’ participation required with respect to the nature of the problem/solution251

• Define the appropriate levels of accountability

• Avoid instability in preferences

243 http://www.whitehouse.gov/open 244 http://www.partner4solutions.gov/ 245 http://www.regulations.gov/#!home 246 http://challenge.gov/ 247 https://wwws.whitehouse.gov/petitions 248 http://changeby.us/ 249 http://wikirendum.org/ 250 http://www.digitalengagement.info/ 251 Making a reliable synthesis of a large-‐scale discussion can be daunting. An approach considered viable is some sort of machine-‐assisted human "harvesting", or "catching": software uses several algorithms to identify possible "atoms of interest". For example, networks analysis can detect balkanization of a community around a polarizing issue; if users give ratings to statements (as is the case in some of the tools examined by CROSSOVER) Bayesian inference on user behavior can detect inconsistencies, and therefore irrational biases. The content thus algorithmically selected is then presented to human "harvesters", who can write summaries. These attention-‐mediation algorithms were apparently developed in the context of medicine, as a tool for helping doctors formulate diagnoses


113 | P a g e

Current research

• Analysing the compatibility of new collaborative behaviour with existing institutional framework

Research disciplines: political sciences, public administration, law, sociology, and other social sciences in general (including institutional economics for example), as well as organisational, network, innovation theories, etc.

Possible research instruments: thematic networks, Support Action



• Updated institutional framework


• New models of governance and service provision

3.2.7. Participatory Sensing Introduction and definition

Participatory sensing refers to the usage of sensors, usually embedded in personal devices such as smartphones to allow citizens to feed data of public interest. This could include anything from photos to passive monitoring of movement in the traffic. Participatory sensing involves higher commitment from citizens, contrary to opportunistic sensing where user may not be aware of active applications. The diffusion of mobile phones significantly lowers the barriers of participation and data input by citizens, with automated geo-‐tagging and time-‐stamping: given the right architecture, they could act as sensor nodes and location-‐aware data collection instruments. While traditional sensor nodes are centralised, these sensors are under the owners’ control. This would give way to data availability at an unprecedented scale.


Participatory sensing radically improves the data availability for evaluating the effect of public policies and how individual behaviour is changing, provided adequate privacy provisions are in place. Devices should assure enhanced users’ control over data, i.e. which data is being sent, when and how it is treated, as well as possibility for enhanced data anonymisation.

Furthermore, design of participatory sensing should be placed in the framework of policy contexts,


114 | P a g e

allowing inference of policy impact from data. Future platforms should combine participatory sensing, mass moderation, personalised feedback and social network analysis to assess the interplay between perception, data and social interaction.

Participatory sensing is already used in “public sphere” activities such as environment and health. However the specific issue of evaluating public policies has been so far little researched, with particular regard to the implications for privacy, large-‐scale deployment and bias management on citizens sensing.

Recent trends

Small-‐scale experiments are being carried out in different domains, mainly dealing with environmental and health data. Applications in the field of urban planning are particularly promising, yet there is no structure link between participatory sensing and policy models. Larger scale deployment would require more granular privacy compliance and user-‐control, adequate incentives to participation and deriving business models. There is no formalisation of the requirements and the design of opportunistic versus participatory sensing, including sampling design for participants recruitment.

Advantage of Application in Public Policy

Participatory sensing can be used to gather and collect the following kinds of information:

• Civic data: neighborhood maintenance issues, power outage documentation

• Environmental data: data providing hints on pollution levels, climate-‐change related data

• Transportation: commutation habits, location and movement data, condition of the road, connections to public transportation, incidence of traffic, accidents occurrence

• Health: vital signs, info providing early warnings of diminishing health, info on epidemic spread, self-‐administered diagnostic tests

The advantages for policy making are:

• Possibility to collect data at an otherwise unachievable scale and geographic range

• Virtually costless data collection

• Reveal and highlight behavioural patterns and routines which can be accordingly changed

• Engage common citizens in sensitive issues

• More pervasive monitoring capacity in fields such as environment and health

Inspiring cases of policy making related applications252

• PEIR 253 (Personal Environmental Impact Report), which is a system allowing users to determine their exposure to environmental pollution by using a sensor in their mobile phone able to determine the location and the mean of transport

• eHealthSense254, automatically detects health related events which are not directly observed by current sensor technology, like pain, tow conditions, depression


115 | P a g e

• SenSay255, which is able to alert the medical staff when the user falls or in case of suspect behaviour

• MobAsthma256, which monitors the exposure to pollution affecting asthma. Both the volume of air inhaled and the pollution rate are collected by sensors interfaced to the mobile phone

• Haze Watch257, in which the concentration of carbon monoxide, ozone, sulphur dioxide, and nitrogen dioxide is measured by embedding pollution sensors in mobile phones

• NoiseTube258, which registers noise levels used to monitor noise pollution, which can affect human hearing and behaviour

• EpySurveyor259, used by the Red Cross to evaluate anti-‐malarial bednet distribution and use throughout sub-‐Saharan Africa, as well as the coverage achieved by vaccination campaigns

• CarTel260, which is a mobile sensing and computing system making use of mobile phones carried in vehicles to collect information about traffic or WIFI access points

• NoiseSpy261, which is a sound-‐sensing system able to log data for monitoring environmental noise. Users can explore a city area while at the same time visualize noise levels in real time


• Preserve the privacy of the users which are required to provide extremely personal data

• Create new mobile device interfaces which are engaging and efficient and can be used

• Ensure security, as the current and past citizen’s position might be spotted

• Provide new sensors capable of increasing the range of information that individuals can track and use

• Create network infrastructures aimed at supporting participatory sensing services

• Provide incentive for participation to data collection

• Develop analytical techniques to carry out more accurate inference with mobile phone supplied data such as geo-‐data and images

• Develop visual analytics and data analysis techniques which provide relevant and easy to interpret information for the general public

• Create engaging and efficient mobile device interfaces to support effective, real-‐time user interaction

• Provide quality data, temporal and geo-‐graphical availability, and ability to cover the phenomena

252 To the best of our knowledge there are not yet government applications in the realm of participatory sensing 253 http://peir.cens.ucla.edu 254 http://dl.acm.org/citation.cfm?doid=1411759.1411761 255 See Siewiorek et al. (2003) 256 http://crystal.uta.edu/~kumar/CSE4340_5349MSE/mobsense.pdf 257 http://www.pollution.ee.unsw.edu.au/ 258 http://noisetube.net/ 259 http://www.episurveyor.org/user/index 260 http://cartel.csail.mit.edu/doku.php 261 www.cl.cam.ac.uk/mobilesensing/downloads.htm


116 | P a g e

Current research

• Aggregating and validating citizens generated and government data resource discovery,

• Selective sharing, and context verification mechanisms, as well as application-‐level support for data gathering campaigns,

• Incentives for participatory sensing,

• Evaluation of human agents as sensors

Disciplines of research: sensor networks, location services; psychology, economics of participation; privacy

Research instruments: testbeds and living labs, STREPs



• Sensing coverage, sensor calibration and sensor context for opportunistic sensing.

• Quality verification for participatory sensing

• Privacy-‐compliant sensing and sharing

• Business models for participatory sensing

• Intelligently recruiting collaborators and deploying data collection protocols.

• Anonymous, transparent use of human-‐carried sensing devices

• Evaluating behavioural change through participatory sensing


• Enhanced analytical techniques to make more accurate inferences from mobile phone-‐supplied data such as location and images and to automatically detect and respond to subtle events;

• New personal-‐scale sensors to expand the range of information that individuals can track and use

• Privacy by design in participatory sensing


117 | P a g e

3.2.8. Identity Management


Digital identity management has long been a policy priority in the EU Member States, and large-‐scale investments have been deployed. In the context of collaborative governance, digital identity constitutes a fundamental pillar of trustworthy cooperation. Identity management systems include control and management of credentials used to authenticate one entity to another, and authorise an entity to adopt a specific role or perform a specific task. Global in nature, they should support non-‐repudiation mechanisms and policies; dynamic management of identities, roles, and permissions; privacy protection mechanisms and revocation of permissions, roles, and identity credentials. Furthermore, all the identities and associated assertions and credentials must be machine processable and human understandable.

At the EU level, the goal is to provide an interoperable privacy protecting infrastructure for eID that is federated across countries, with multiple levels of security for different services, relying on authentic sources, and usable in a private sector context.

Alongside this, a flexible, context-‐dependent and interoperable identity management system is required for large-‐scale deployment. In particular, federated identity management systems that ensure flexible deployment and seamless integration of users’ preferred identities, including commercial (such as Facebook connect) and open source solutions (such as OpenID) are needed. Particular focus should be put on usable delegation of privileges, which is very important for workflows and integrating services.

Electronic identity management should identify non-‐humans (devices, sensors) as well as humans, in order to ensure validated identity in the context of participatory sensing and the Internet of Things.

At the same time, eIdentity management should take into account the risks of information centralization in terms of data privacy and security. Cost-‐benefit considerations of centralised versus federated systems remains a key issue. Identity federation can be accomplished in any number of ways, some of which involve the use of Internet standards, such as the OASIS Security Assertion Markup Language (SAML) specifications, with the use of open source technologies and/or other openly published specifications.


Identity certification is one of the core tasks of government, and therefore pertains specifically to the governance context. This is reinforced by Meta Group (2002), who views the implementation of identity management “not as a differentiator but as mandatory security consideration, a business imperative and a non-‐negotiable user expectation”.

Recent trends

The role of Identity Management is vital in the context of ICT for Governance and Policy Modelling. The importance of addressing eIdentity-‐related issues for secure public service provision, citizen record management and law enforcement has made Identity management a strategic issue for governments at both a local and international level. Research for the design and implementation of privacy preserving digital identity, as well as for its supporting management infrastructures, and delegation of authority, has reached a satisfactory level. Nevertheless, one of the greatest problems in Identity Management is lack of interoperability of digital identities and identity management systems between proprietary systems and standards-‐based ones, and between organisations and governments.


118 | P a g e

Current practice

• Electronic ID creation at national level

• Pilots in cross-‐border interoperability of field in EU (STORK project)

Public Policy Applications

The development of Federated Identity Management would be to the following benefits at governmental level:

• Avoid replicated efforts: reduction in the number of sign-‐ons and passwords needed for accessing multiple systems and databases, thereby decreasing cost and time-‐waist

• It would be possible to define a mechanism of sharing and managing identity information as it moves between discrete legal, policy and organizational domains which would be based on standards

• Institutions would not have to establish separate relationships and procedures with one another

• It is possible to grant ad revoke user access to info more easily

• Reduce the number of passwords accumulated: citizens either forget them or choose simple ones thereby increasing insecurity and fraud possibility

• Increase in security regarding the user access to information and the digital resources, as it eliminates the need to replicate databases of user credentials for separate applications and systems, which are potential weak points

• Increase in sensitive information shared across government and organizational boundaries in case of crisis

• Allows to focus on users of information and services rather than on entities that house those resources


• Fragmentation of research in identity along disciplinary lines

• Need for new identity proof processes

• Privacy issues: use limitation principles, avoid pervasive surveillance

• Capability to efficiently integrate services throughout the chain

• Time saving identification

• Specifications and nature of a Digital Identity dictated by the social and political environment of the country of issuance

• Increasing number of electronic identity-‐related crimes (identity fraud, identity theft, impersonation), which makes it difficult to guarantee the legitimacy of identities

Current research

• Cultural-‐dependent identity systems

• Mobile and biometrics in eIdentity

• Privacy protecting identity management systems

• User-‐centric identity, delegation of authority


119 | P a g e

Disciplines of research: legal, technological, social, economics

Possible research instruments: testbeds and living labs, STREPs



• Quantitative research on cost-‐benefit analysis of interoperable identity

• Dynamic user-‐controlled identity disclosure

• Formal verification of identity management systems

• Governance and legal issues, levels of assurance


• Context-‐dependent identity management


120 | P a g e

3.2.9. Global Systems Science

Introduction and definition262

Current tools available to policy makers are insufficient for providing guidance on a global scale in facing present societal challenges because of the connections across subject domains as well as the globalization of the policy challenges, which range from environmental threats, food security, or energy sufficiency. Such challenges are multi-‐dimensional and borderless, thereby they cannot be solved by one single country or by one aspect of policy. In fact current public policy making is targeted at individual, rather than interrelated systems, and thereby struggles in achieving systemic change and in addressing challenges which are global and interconnected in scope, as they arise from the interplay of social, technological, and natural systems. In this respect it is important to integrate scientific evidence into social process for being able to address those challenges.

In this view we need a new multidisciplinary system approach taking into account the connections across policy areas (e.g. economy, transport, health and social understanding of system risk) as well as across geographical borders. This new branch of science should take into account the multidimensionality of global problems given by the interconnectedness of decisions across different policy realms.

As stated in the Cordis website263, Global System Science “addresses new ways of supporting policy decision making on globally interconnected challenges such as climate change, financial crises, or containment of pandemics. The ICT engines behind GSS are large-‐scale computing platforms to simulate highly interconnected systems, data analytics for 'Big Data' to make full use of the abundance of high-‐dimensional and often uncertain data on social, economic, financial, and ecological systems available today, and novel participatory tools and processes for gathering and linking scientific evidence into the policy process and into societal dialogue. GSS will develop further the scientific and technological foundations in systems science, computer science, and mathematics.”

Some examples of global systems are the following:

• Energy, water and food supply systems

• Community of scientists

• World wide web

• Globally spreading diseases

• Global financial system

• Climate policy

• Web of military forces and diplomatic relations

262 Among the sources of this research challenge there is the position paper “GSS: Towards a Research Program for Global Systems Science” prepared for the Second Open Global Systems Science Conference which took place on June 10-‐12 2013 in Brussels, as well as other documents related to the GSDP consortium 263 http://cordis.europa.eu/fp7/ict/fet-‐proactive/fetconsult2012-‐topic09_en.html


121 | P a g e

ICT tools are very important for GSS, as they support large-‐scale, complex societal and infrastructure decisions that affect human life and health. On the side of policy informatics, they provide a scientific evidence-‐base for policy, i.e. models and data integrated across different policy sectors. On the side of societal informatics, ICT tools, presenting the models’ results by the mean of games and visualization, are able to integrate the can make a better link between stakeholders in the scientific and policy process, leading to a society-‐centred science. In this sense ICT tools are also very useful to solve problems for policy makers, engage domain experts and empowered officials early, and demonstrate relevance of the treated issues.

Why it matters in Governance

There are two main motivations for adopting a Global System Science approach:

• Develop new models in support of decision-‐making: obviously nowadays the most important issue to deal with lies in the economic and financial crisis. A typical example of global systems model is given by the interdependencies among the actors and institutions in the financial system as well as the contagion channels between the financial systems and the other sectors of the economy. Having neglected or overseen such interconnections has led to dangerous chain reactions and contagion of crises not anticipated in current economic models. In this respect global systems models can give an alternative to existing theoretical and empirical approach, in particular when facing challenges which are global by definition, such as financial instability and the environmental or energy policy. The point of arrival will be the definition of advanced simulation models mimicking factual conditions and human behaviours, and embedding empirical data on systemic dependencies as well as on the role of human behaviour. Such models will be funded on large-‐scale agent-‐based modelling, will allow stakeholders participation and interaction and online monitoring with feedback from individual citizens.

• Develop new models of governance: there is the necessity for scientific modellers to better communicate and interact with citizens, businessmen, politicians, civil servants, NGO representatives and other stakeholders as concrete societal needs and policy decisions must drive the scientific questions to be asked, the data to be collected and how the models have to be conceived. Global Systems Science, by producing better models, can provide the decision making process with insight on system behaviour and dynamical outcomes, leading to better policies. In this respect the current global governance model which is based on nation states cooperating in international organizations is unfit to meet global challenges, so that a Global Systems Science is necessary.

Tools and Techniques in GSS

GSS includes in general computer science and mathematical approaches such as interaction based computing; data topology and modeling languages; high performance computation; data mining methodologies; methods for specification and analysis of dynamics of highly interconnected systems; specification, verification and validation of the computational dynamics simulations, and formal approach to the analysis of dynamical network abstractions for complex system representation. More in particular we have:

• Agent-‐based or Multi-‐agent Models, which are synthetic virtual realities populated by


122 | P a g e

artificial agents interacting adaptively with each other and also change with the overall conditions in the environment

• Analyses of networks based on maps of relationships or linkages among constituents in systems, from which is possible to identify configurations that are especially unstable and can be used as predictors of catastrophic failures in real-‐life networks

• Data Mining: techniques for finding patterns and relationships in large data sets with complex qualities, which are applicable to nonlinear and discontinuous phenomena

• Modelling of artificially constructed scenarios representing hypothetical models of complex systems that reflect their key constituents and dynamics, and which can be used to anticipate the effects of various conditions and to identify policies that are robust to many likely futures, for instance in case of man-‐made or natural disasters

• Sensitivity Analysis, used for assessing the behaviours and evolution of complex systems due to shocks in the underlying parameters performed by the mean of numerical techniques

• Dynamical Systems Models, i.e. sets of differential equations or iterative discrete equations used to describe the behaviour of interacting parts in a complex system, and used for simulate the results of alternative system interventions as well as unintended consequences of policies

Policy Applications of GSS Tools

Health. In contrast with traditional epidemic models, in which each agent bears the same probability of infection, agent based models entail a heterogeneous population which interacts in a changing environment leading to more realistic tests and prediction of new policies. As an example some complexity-‐science simulations showed that reductions in air traffic (even 20%-‐50%) would not dramatically slow the spread of certain epidemics. On the other hand the massive storage of smallpox vaccine would reduce the number of infections in case of a biological terror attack.

Urbanization. More complex patterns displace the classic centre-‐periphery structures and puts into question the distinction of nature and culture. Moreover urban lifestyles are blended with the global awareness fostered by ICT. In this view urbanization raises major challenges because innovations my worsen already worrying trends, urbanization can undermine communities leading to new forms of violence and anomie, and health problems such as circulatory diseases, cancer, obesity and epidemics can be augmented. GSS is able to explain how settlement structures and lifestyles are modified by interaction between the global urban system and the global ICT system, as well as how policy-‐makers can influence their future dynamics.

Traffic. Analytic techniques can be used to anticipate life-‐threatening traffic phenomena, as well as to reduce pollution and improve traffic flows in order to save time and energy. This advanced modelling approach incorporates human cognition and has been adopted for predicting unexpected events such as traffic jams so as to automatically alert drivers via wireless communications devices. This class of models can be generalisable to other types of situations, e.g. outbreaks of civil unrest.


123 | P a g e

Similar methods have been successfully used for analysing human foot traffic and have been applied to prevent stampeding during the Hajj in Mecca.

Crisis management and security. By using a global systems science and complex systems it is possible to enable bottom-‐up disaster response capabilities as well as to implement a proactive approach to disaster preparation and planning, by the mean of policy simulations. Moreover network-‐analysis methods can be used to attempt to identify associations of terrorists, including pinpointing the locations of key dangerous individuals.

Climate Change. In the most advanced climate change models is missing the social and human aspect of the issues given by the strict interconnection between nature, economy, finance, energy and the industrial structure. In this respect complexity and global systems science techniques allow to identify tipping points in the human-‐earth system. An example is given by the management of water resources: water stresses occur regularly in different geographical locations, showing that a tipping point, leading to massive long-‐term water shortages, may be close. GSS will support global climate policy by highlighting its benefits from reduced health impacts to accelerated productivity growth by new directions and volumes of investment, in order not to be stuck in multiple basins of attraction. This will require new models as well as greater interactions between different policy fields such as environment, energy, employment, health and foreign policy. Policy makers will join to show that increased economic well-‐being is possible with systematically decreasing emissions, strengthen resilience while reducing emissions, and prepare for the need to take CO2 back from the atmosphere, especially once global poverty will have been overcome.

Financial Markets. Decision support and analysis tools, based on modelling and simulation, conceived within the scope of global systems science models can allow the theoretical testing, of the resilience of proposed financial regulations in order to avoid the dramatic instabilities of recent times. Those classes of models can offer a crucial supplement to traditional analysis as they emphasize dynamism rather than equilibrium, real attractors rather than theoretically prescribed ones, positive feedback loops and phase transitions. The current financial crisis did not crashed completely the Euro economy thanks to the president of the ECB, Mario Draghi, who pushed the market from a bad to a good equilibrium rather than considering the crisis as a shock that had to be absorbed by the capacity of the markets to return to the stable equilibrium. In the next decades GSS will be necessary for the development of an integrated governance of global risks taking into account the interactions between financial and other markets, as well as the socioeconomic dynamic at different scale, relying on the analysis of the large data-‐sets for monitoring the complex networks of world agents. Researchers and policy makers should join within the scope of GSS in order to design and implement effective measures towards a financial sector supporting increasing employment and sustainable economic growth, such as rules to limit risky dynamics of complex financial systems, regional experiments with innovative schemes to foster sustainable growth, and the creation of a global monetary system.

Methodological Aspects

First of all GSS will rely on computer models in order to tackle the complex multi-‐scale (spatial and


124 | P a g e

temporal) structure of global systems. However computer models are ill suited to deal with the ambiguities that are a vital ingredient of human life. In this respect narratives delimit the scope within which a particular model is useful and understand what goes wrong when it is used beyond that scope.

Another interesting methodological realm of GSS is high performance computing (HPC), which should provide policy-‐makers with a scientific evidence coming with an assessments of its reliability, validity, and relevance, by exploring complex sample spaces of parameter values and boundary conditions. To this respect the computational skills must be combined with great skills in communication and in assessing the relevance of evidence for addressing specific practical issues.

Moreover Big Data, if used in new ways, can become an essential tool to perceive global systems. GSS can define practical problems and preliminary concepts that can be used to mine big data sets, which can be obtained by crowdsourcing, in the view of describing the dynamics and structure of global systems, by exploiting the relation between models and narratives of globalization. The resulting output will be used to improve problem definitions and concepts, as well as to monitor the intended and unintended consequences of policies dealing with global systems.


GSS entails a number of challenges and gaps:

• Model validation in terms of the underlying assumptions and parameter choices interfaces

• Creation of ICT platforms for setting up, execute and validate large-‐scale models. In particular it would necessary to establish agreed-‐upon standards for validation and calibration in order to improve the models credibility for policy makers

• Tools for gathering, integrating and linking data from various sources: financial data, socio-‐economic data, data on financial and economic networks, ecological and energy data, and even data on nature of human decision. Tools for knowledge elicitation

• Decision-‐support tools: scientists and researchers should try to formulate the results of their work in terms that policymakers can use, for example through simulations showing different scenarios in a global setting

• Interoperability of Models: models should be built in order for the results of modelling to be comparable, eliminating the heterogeneity preventing non-‐experts from choosing and applying models, as well as gauging their relevance and credibility. Moreover we should be able to run simulations in different degrees of granularity

• Institutional adaptation: the development of a science which is global in scope requires coordination of research and education efforts at a global level

A more comprehensive set of challenges and opportunities can be found at http://goo.gl/qWbhG264. Here we summarize the main points:

• Breaking inter-‐disciplinary boundaries, stimulating cross-‐disciplinary fertilization and removing silos between organisations, governments, scientists/technologists and other

264 This set of challenges and opportunities has been developed by the experts conveyed at the brainstorming meeting on "Global System Sciences: the role of models and data", which took place in Brussels on the 7-‐8 February 2013 http://www.isi.it/events/workshop-‐on-‐global-‐systems-‐science-‐role-‐of-‐models-‐and-‐data-‐brussels-‐february-‐7-‐8-‐2013


125 | P a g e

stakeholders • Building the scientific foundations of GSS, scoping the technologies and methods,

changing basic paradigms in science and making society and humans parts of the phenomenology of science

• Managing and making sense of Big Data: manage the increased input of data/information and analyse big data while preserving ethical values and respecting privacy and civil rights

• Models and simulation, languages: create a common universal language to precisely describe and simulate models, close the gap between formalized models and real world peculiarities of systems

• Infrastructures and resources: develop a culture of ICT as the fabric that connects nature and society, and push a new conceptual advance in order to overcome the limits of computing

• Ownership and regulations: enable scientists to act on society and to access the data without violating individual rights to privacy, and manage copyright/IPR in the hyper connected world

• GSS and policy making: ensure take up in GSS from decision makers and policy makers, enable policy feedback to be reflected into the models and create tools for effectively support decision making

• Communicating GSS and engaging society: raise awareness of Complex Systems science and promote GSS skills/curricula

As for the use of ICT tools in GSS there are several categories of challenges265:

Evidence Dissemination Global

Reasoning Proposal Governance Implementation

• Scientific data • Scientific code • Complex

modelling and

simulation

• Software architecture,

sustainability,

interoperability,

interfaces

• Common data

interchange

format

• Domain

specific

languages

• Visualization • User interface • Independent validation

• ICT tools for learning and

understanding

• Accessibility • Tools for interactive

participation

• Computer

linguistic for

narrative and

automatic

translation

• User-‐friendly

simulation

tools for

prediction

• Computer

security for

trust

management

• ICT platforms

• ICT tools for transparency

and

participation

• Semantic

frameworks

for computer-‐

added decision

making

• Use of ICT platforms

• Crowd-‐sourced development

• Feedback to the scientific data

management and the

model

265 Source: readapted from the presentation delivered by Ulf Dahlsten at the First Open Global Systems Science Conference (Brussels, November 8 – 10, 2012). A Tech4i2 delegate attended the meeting. http://blog.global-‐systems-‐science.eu/?page_id=938


126 | P a g e

• Validation

Table 2 Challenges in the use of ICT tools for Global Systems Science

Current Research

• Computer Science for interacting Informational, Technological and Social Networks • Computer Science of very large systems: high performance computing and data-‐driven societal science • Advanced computing for Network Science • Network Science as an integrating framework for real world complexity • Network approach for governance and policy tools • Computational modelling in complex realities • Computational and digital epidemiology • Mathematics of complex systems

Future Research

Nowadays there is not a mathematical model able to describe all the interactions between the components in Global System Science. So far scientists have implemented components and their interactions with code, but there is the need to create an intermediate, mathematical layer between narratives and simulations, such as in physics. This mathematical layer cannot be based on mere partial differential equations or functional analysis. On the contrary, as the formal language of GSS is computer code, the mathematical layer has to be embedded in the mathematics of general programs. In practice computer science should play for GSS the same role that mathematics plays for physics. In this sense there is the necessity to adapt and extend to the GSS models one or more of the formal languages for specifying and reasoning about programs. The main candidate so far is the constructive type theory, which can be used to express both programs and classical mathematical results.


127 | P a g e

4. The case for policy-‐making 2.0: evaluating the impact These technologies and methodologies certainly show great promises for better policy-‐making, but to what extent do they genuinely lead to better policy-‐making?

In this section, we provide an overview of the evidence regarding the actual impact of policy-‐making 2.0 tools. To do so, we extract the main related findings from the cases studies; from the prize on policy-‐making 2.0 launched by the project; from the survey of users’ needs. Based on the findings, we propose an additional research challenge on the impact evaluation of policy-‐making 2.0.

4.1. Cross analysis of case studies

In deliverable D5.2, the CROSSOVER project provides an in depth analysis of 4 cases studies:

1. Gleam

2. Pathways 2050

3. UrbanSim

4. Opinion Space

In this section, we extract and analyse the relevant information about their impact on the quality of policy-‐making.


128 | P a g e

4.1.1. Global Epidemic and Mobility Model

GLEAM -‐ The global epidemic and mobility model266, is a discrete stochastic epidemic computational model based on a meta-‐population approach in which the world is defined in geographical census areas connected in a network of interactions by human travel fluxes corresponding to transportation infrastructures and mobility patterns. The GLEAM 2.0 simulation engine includes a multi-‐scale mobility model267 integrating different layers of transportation networks going from the long range airline connections to the short range daily commuting pattern268 and it elaborates stochastic infectious disease models to support a wide range of epidemiological studies, covering different types of infections and intervention scenarios in order to respond to the spread of a pandemic crisis in very short time. Real-‐world data on population and mobility networks are used and integrate those in structured spatial epidemic models to generate data driven simulations of the worldwide spread of infectious diseases.

GLEAM is mostly used in the design stage of the policy making cycle, and it is able to manage and visualize with a huge amount of complex and diverse data (e.g. detailed airline transportation model). In fact, data from census agencies, data regarding population at very high resolutions, data from a world map implemented by NASA with the world population divided to 5x5 miles area boxes, the entire database of airlines, about 40 databases from different countries for local mobility, transfer etc. are utilized. In addition, it has to be mentioned that GLEAM has moved beyond research in the H1N1 epidemic case; when the simulation derived from the application of GLEAM was used ex-‐post and resulted in a particularly accurate analysis. GLEAM is nowadays utilized both in research initiatives (e.g. EPIWORK IP project269, EPIFOR project270) and in formal policy making agencies (e.g. US Defense Agency). Moreover, GLEAM can also be met in educational courses; both in a high school and at the university level.

Figure 4: The three population and mobility data layers in GLEAM

Impact of Gleam

The main impact of GLEAM so far was the production of the forecast for the H1N1 pandemic in real-‐time which was a quite successful exercise and showed the power of the model. A validation paper (Tizzoni et al. 2012) has been published in December 2012 showcasing that the GLEAM predictions were quite spot on.

266 http://www.gleamviz.org/ 267 http://www.gleamviz.org/model/ 268 GLEAM in Detail. Available at: ww.GLEAMviz.org/GLEAM-‐in-‐detail/ 269 EpiWork -‐Developing the framework for an epidemic forecast infrastructure. Available at: http://www.epiwork.eu 270 EpiFor -‐ Complexity and predictability of epidemics: toward a computational infrastructure for epidemic forecasts.

Available at: http://www.epifor.eu


129 | P a g e

Many stakeholders are also using the software and support their policy-‐making procedures in terms of designing measures to prevent or constrain the spread of diseases. Examples include the US Defense Agency, the JRC, and other corporations that are using the software. It has to be noted that JRC is using the tool in its long-‐term strategy for studying and responding to the spread of epidemics (through communicating the simulation results to DG SANCO policy officers), based on the experience that has been accumulated from using the GLEAM toolkit during the H1N1 disease.

The core innovation of GLEAM lies within the computational model which can integrate data from various sources and provide a close to real time forecast (by combining various real-‐time data sources) on the spread of epidemics on a global level, which was not possible before at that level of precision and punctuality. Moreover, through the visual interface users are in a position to create their own models and investigate specific diseases and issues that they are interested in.

4.1.2. UrbanSim

UrbanSim271 is a software-‐based demographic and development modelling tool for integrated planning and analysis of urban development, incorporating the interactions between land use, transportation, environment, economy and public policy with demographic information. It simulates in a 3D environment the choices of individual households, businesses, and parcel landowners and developers, interacting in urban real estate markets and connected by a multi-‐modal transportation system. The 3D output resulting from the process underpinning the simulation model is presented using indicators, which are variables that convey information on significant aspects of the simulation results.

Figure 5 UrbanSim Land Maps

This approach works with individual agents as done in agent-‐based modelling, and with very small cells as in the cellular automata272 approach, or even at building and parcel levels. UrbanSim

271 http://www.urbasim.org 272 http://en.wikipedia.org/wiki/Cellular_automaton


130 | P a g e

however differs from these approaches by drawing together choice theory273, a simulation of real estate markets, and statistical methods in order to achieve accurate estimation of the necessary model parameters (such as land policies, infrastructure choices, etc.) in order to calibrate uncertainty in its system. As an example of its use, one could refer to the project on Modelling Land Use Change in Chittenden County274, where the model parameters based on statistical analysis of historical data are integrated with market behaviours, land policies, infrastructure choices in order to produce simulations on household, employment and real estate development decisions (where the first two are based on an agent-‐based approach while the latter on a grid-‐based approach).

Impact of UrbanSim

As far as the impact is concerned, the European case is not at the same level as the US ones. In the US there are quite a number of MPOs that actively utilize the UrbanSim platform. The most indicative application, representing the approach common in the US, is probably the San Francisco Bay one. The results of the aforementioned case have involved examining and analysing five alternative scenarios that required articulating a set of assumptions about land use policies, transport policies and macro-‐economic growth (the analysis in now complete – relevant publications will be available in the next few months).

In one of them, analysing visibility of the proposed policy though reverse engineering was attempted, that made the task much more challenging, both in terms of research and implementation. The agency has now accepted the results, with documentation and visualization supporting them.

In the San Francisco case, the 3D visualization system was created in order to achieve higher visibility amongst citizens than the plain UrbanSim tool. The intention was to use this system in a number of workshops held during January 2012. User engagement was intense even from the development/testing phase. In addition, the public agencies used it in a series of meetings with community organizations. Each of these meetings had from 15 up to 200 participants each. The point of these meetings was to communicate the different scenarios to the public and to receive feedback on the preferences of the citizens.

One of the most innovative elements of UrbanSim is the combination of various technological and theoretical aspects, as well as the withdrawal of strong assumptions regarding urban planning and adoption of less strong assumptions (than markets are an equilibrium). For example, the impacts of transport projects on urban planning are far from being instantaneously realized (in fact they might evolve over decades). In addition, the capacity of being able to support these less strong assumptions can also be considered as a core innovation.

4.1.3. Opinion Space Launched by the U.S. Department of State275 in collaboration with Berkeley University which developed it, Opinion Space bridges the worlds of politics and social media in an interactive visualization forum, where users can engage in open dialog on foreign affairs and global policies. It invites users to share their perspectives and ideas in an innovative visual "opinion map" that will illustrate which ideas result in the most discussions and which ideas are judged most insightful by the community of participants.

273 http://en.wikipedia.org/wiki/Choice_theory 274 http://www.uvm.edu/rsenr/countymodel/Workshop08bv3.ppt 275 U.S. Department of State. Available at: http://State.gov


131 | P a g e

Using an experimental gaming model, Opinion Space incorporates techniques from deliberative polling, collaborative filtering, and multidimensional visualization. The result is a self-‐organizing system that uses an intuitive graphical "map" that displays patterns, trends, and insights as they emerge and employs the wisdom of crowds to identify and highlight the most insightful ideas.

Figure 16 Rating other opinions' in Opinion Space

Opinion Space is fully operational in its current state. Nevertheless, as a research platform it still remains experimental. The great amount of data is very structured and this helps towards continuing research on text analysis, statistical modelling etc.

Impact of Opinion Space

One of the first and main indicators of the impact of Opinion Space, which have applies mostly to the “agenda setting” and “monitor and evaluation” phases of the policy cycle, was the participation rate: users that arrive in the platform for the first time and those that become active participants. People that arrive in websites are always more than those who actually participate (in some projects the rate was close to 50% and in others around 10%).

In the State Department instance (of Opinion Space 3.0), more than 2000 different ideas were collected (about US foreign policy). In addition, more than 5000 individual responses were collected. It cannot be said whether the final decisions were based on some of the ideas provided, but a detailed report was provided to the policy makers. The project with a US auto-‐maker (targeted towards recognizing ways of improving their image) resulted to about 1000 ideas and about 100.000 ratings evaluating these ideas (e.g. more specifically they talked about green vehicles). One of the core innovations and successes of Opinion Space is the very fast way to browse (and rate) amongst a large number of ideas (even if this is a visualization-‐oriented innovation). From the scientific point of view, the greatest innovation was bringing statistical analysis in structured discussion/ data.

One of the best endorsements regarding Opinion Space was Hillary Clinton’s reference to the initiative. Other endorsements include high level officers of collaborating companies as presented in the Opinion Space website. As far as the Opinion Space team is aware of, Opinion Space has not yet


132 | P a g e

been incorporated in any formal decision making procedures. The State Department, however, uses “informally” Opinion Space in order to get ideas and opinions on specific policies.

4.1.4. 2050 Pathways Analysis The UK Department of Energy and Climate Change (DECC) built the 2050 Pathways Analysis Calculator to help the public engage in the debate, and for Government to ensure that its short-‐ and medium-‐term planning was consistent with achieving the long-‐term aim. More specifically, as the UK is committed to reducing its greenhouse gas emissions by at least 80% by 2050, relative to 1990 levels, a transformation of the UK economy is needed while ensuring secure, low carbon energy supplies to 2050, and face major choices about how to do this. In the Carbon Plan published in December 2011, the Calculator was used to illustrate three 2050 futures that show some of the plausible routes towards meeting the target.

The 2050 Pathways Analysis features four resources:

1. A web-‐based tool for the public to try their own ideas for reducing greenhouse gas emissions.

2. An in depth Excel-‐based tool and reporting system which includes the methodology/the models that are used for the analysis.

3. A web-‐based presentation for younger audiences about greenhouse gas emissions.

4. A toolkit for leading an energy debate in schools.

The 2050 Calculator is targeted at citizens, policy makers, senior officials and politicians as well as technical experts through different interfaces.

The 2050 Pathways presents a framework through which it is possible to consider some of the choices and trade-‐offs we will have to make over the next forty years. It is system-‐wide, covering all parts of the economy and all greenhouse gases emissions released in the UK. It is rooted in scientific and engineering realities, looking at what is thought to be physically and technically possible in each sector276.

2050 pathways is a tool to help policy makers, the energy industry and the public understand these choices. For each sector of the economy, four alternative trajectories have been developed, ranging from little or no effort to reduce emissions or save energy (level 1) to extremely ambitious changes that push towards the physical or technical limits of what can be achieved (level 4).

The 2050 Pathways Calculator – available on the DECC website -‐ allows users to develop their own combination of levels of change to achieve an 80% reduction in greenhouse gas emissions by 2050, while ensuring that energy supply meets demand277.

The supportive tools of the initiative provide different ways of securing a low-‐carbon future for the UK and they can be tried out:

·∙ By creating each user’s own pathway using the 2050 Web Tool.

276 Department of Energy and Climate Change https://www.gov.uk/2050-‐pathways-‐analysis 277 HM Government (2010). 2050 Pathways Analysis. Available at: http://www.decc.gov.uk/assets/decc/what%20we%20do/a%20low%20carbon%20uk/2050/216-‐2050-‐pathways-‐analysis-‐report.pdf


133 | P a g e

·∙ By exploring what a low-‐carbon UK might look like in 2050 by playing the simplified My2050 simulation.

·∙ By taking the debate into the classroom in the schools toolkit.

Figure 17 Playing the My2050 game for the demand side

As far as the CROSSOVER Policy Cycle is concerned, the project probably fits in the first step, this of Agenda Setting. This is due to the fact that the concept is a high-‐level one (e.g. reduce gas emissions to 80% by 2050). As the data are currently being updated and a comparison between the projected and the actual results will take place, probably the case could in the near future fit into the Monitor and Evaluation Policy Cycle step as well.

Impact of 2050 Pathways Analysis

The numbers of visitors and of interactions with the tool have demonstrated the success and impact of the case. In the first three months from the official project launch there were about 10.000 unique visitors in the platform. Regarding My2050 there are over 16.000 pathways up to the date. Regarding the stakeholders, about 200 were involved in the initial (building) phase and after the launch about 500 stakeholders were contacted. Moreover, a week-‐long online debate including 5-‐6 experts took place with lots of comments from open public. It is important to note that there are Master’s programs, both in and outside of the UK, that engage the 2050 Pathways models and tools in their courses. In addition, the my2050 game is also communicated to pupils of various schools in the UK; there is a “schools’ toolkit” available and downloadable from the project’s website, as well as from other websites, including the department of Education website.


134 | P a g e

It has to be noted that due to the project’s open source nature, it is quite difficult to tell how many and who exactly are using the platform.

In addition, a large number of presentations have been conducted in workshops, schools, conferences, NGOs, international colleagues etc. A presentation was made to the European Commission too. Really positive media coverage has also been noticed (around 15 key articles regarding the project278279). Other references to the case have also been made (e.g. cultural festivals).

4.1.5. Cross analysis of the case studies In this section we analyse common features and differences between the case studies with regard to:

-‐ Usage (policy phase, policy domain, participation, involvement of decision-‐maker)

-‐ Impact (satisfaction, role in the actual decision taken, quality of the policy)

278 https://www.gov.uk/2050-‐pathways-‐analysis 279 http://www.involve.org.uk/2050-‐pathways-‐public-‐dialogue/


135 | P a g e

2050 Pathways GLEAM Opinion Space 3.0

UrbanSim

Policy phase Design Design Agenda Setting Design

Policy domain Energy Health Foreign Policy Urban Planning

Number of participants

16.000 pathways created

Not relevant 2000 ideas 100s

Involvement of decision makers

High High High High

Actual usage of the output in policy-‐making

High (used in the main Low Carbon Strategy document)

High, used by international agencies

Low Medium, used by several US municipalities

Press impact High Low High n.a.

Feedback by policy-‐makers

High n.a. High n.a

Actual improvement of policy quality

n.a. 1 paper positively reviewed the predictions

n.a. n.a.

Table 3 Cross analysis of the cases impact

Firstly, we can perform a matching of the cases under scrutiny with respect to the different phases of the policy cycle. As we can see from Table 3 Cross analysis of the cases impact

most of the cases are related to the “Design” of policies, while there is a limited coverage of the “Agenda Setting” and the “Implementation” and “Monitor and Evaluation” phases280.

This is due to the fact that the key challenges faced by the policy makers (e.g. “the need to detect and understand problems before they become unsolvable” or “the reduction of uncertainty on the possible impacts of policies”) require a certain degree of proactivity in order to deliver high quality, evidence-‐based and impact oriented policies and not perform trials on real conditions. In this respect the “Design” phase seems to prevail over the others when it comes to tools that are mostly desired by policy makers. More in particolar:

• In the “Design” phase policy makers are able to both explore their options and seek for the ex-‐ante assessment of the policies under consideration from the citizen’s perspective. On the other hand it is possible that decisions have been already taken and then the emphasis is laid on the implementation of policies.

• In the “Implementation” phase the main object is to increase acceptance and collaboration between the decision makers and the citizens based on already deployed terms. On the other side it is worth noticing that the improved collaboration is handled by tools and methods focusing on the communication of messages aimed at favoring the smooth implementation of a policy. Those tools do not belong to the “core” Policy Making 2.0 methods, even though they

280 X’s marks the answers retrieved directly for the responsible team of each case, while @’s mark potential usage as envisaged during the analysis


136 | P a g e

display a close relation with them.

• In the “Monitor and Evaluation” step decision makers get informed about the impact of the already deployed policies, so that it is possible to identify only few ICT-‐based tools and methods that are really having an impact and engage fruitfully with stakeholders and citizens. There are many discussion tools, which have already been experimented by policy makers, showing so far many limitations and constraints.

• The issues apply also to the “Agenda Setting” phase, as there is an absence of new ways to massively engage citizens during the early procedures that lie before the actual design phase. Most of the tools have been around since many years now, and in some cases are merely re-‐furbished with some new tweaks and upgraded features. Crowdsourcing seems to fit very well this stage, but again the impact of such experiments remains anecdotal and the results are not embedded in policy-‐making, at least for the cases analysed.

In these cases, the policy-‐making 2.0 tools have been used to address real problems in sensitive policy domains, and all have been initiated either by governments or as a result of collaboration between researchers and public administrations at different levels, mainly in a top-‐down approach. In particular, GLEAM and Opinion Space 3.0 were initially introduced as research initiatives that gathered significant attention and subsequent funding from public authorities. In fact, all cases build on a wide range of techniques that result from research and exemplify how research can be effectively applied in real-‐life settings and public policies. Multi-‐disciplinarity in the teams of all cases has brought together different perspectives and ensured appropriate modelling of policy options and interpretation of outcomes. Building a dynamic dialogue with policy makers and all external stakeholders (NGOs, academia, industry) and specific experts, has provided significant insights and feedback to all cases (to different extents as for example in GLEAM, where the participation of citizens is limited). Further, the real support by public officials and experts has been instrumental in the success of all cases. To address the targeted needs of policy makers and citizens and allow them contribute in a more efficient and productive way to the policy issues at stake, dedicated tools have been developed in each case study. Naturally, in each case, the required learning curve to understand and use a policy model significantly varies (and it depends on the complexity of the policy model(s) running in the background for being used effectively by policy makers).

Uptake by participants varies, from few hundreds up to several thousands. Impact evaluation, however, was not built-‐in the initiative from the beginning. Typically, no specific Key Performance Indicator (KPIs) were set, and no evaluation envisaged. However, the numbers of visitors and of interactions have demonstrated their success and impact, which has been reinforced with the help of appropriate stakeholders’ engagement strategies. It needs to be noted that in some cases (GLEAM) users resorted to the corresponding platform as a result of a natural phenomenon (i.e. H1N1 pandemic) whereas in others (Opinion Space 3.0 and 2050 Pathways Analysis), it was the outcome of large press coverage that demonstrated the value of the cases. By studying cases that had strong internalization aspects (i.e. transferring experience from national to international level in 2050 Pathways Analysis, from US to EU in UrbanSim), the difference in socio-‐cultural dimensions emerges and should not be neglected as it may decide the success of a case in applying it to different geographic settings and socio-‐technical landscapes.

4.2. Survey of Users’ needs results

The CROSSOVER project delivered a survey of users, presented in Deliverable D5.1. As part of the survey it was asked which ICT tools and methodologies are needed and adopted by respondents as


137 | P a g e

part of the governance and policy-‐making processes they are involved in. In Figure 2, which displays some preliminary and selected results, it emerges that Open Data and Big Data methodologies are already adopted by more than 30% of respondents. Moreover, other methodologies, which are also used, are strictly related to Open and Big Data, such as visual analytics that can be used to make sense of large amounts of data, and large scale simulations which need large amount of data to be performed.

Figure 18 Adoption of ICT Tools and Methodologies for policy-‐making (source: CROSSOVER Survey of Users’ Needs 2012)

In the same way in Error! Reference source not found.3 are presented the respondents’ views regarding the needs and challenges in the policy making process.


138 | P a g e

Figure 19 Needs and Challenges in the Policy Making Process (source: CROSSOVER Survey of Users’ Needs 2012)

On the horizontal axis is reported the average score accruing to each option. The score goes from 1 (not important) to 5 (very important). As we can see the most relevant challenges in the policy making process are “Detect and Understand Problems before they become unsolvable” and “Understand the Actual Impact of Policies”. At any rate the difference among the various options does not seem overly significant, suggesting that the broad range of challenges in policy making is recognized as important.

The respondents were also required to suggest other important challenges pertaining to the policy making activity, outside the one adduced in the online questionnaire. The suggested challenges include:

• Create an effective and collaborative dialogue among the policy makers and affected stakeholders

• Ensure reversibility as well as basic societal values (e.g. security, equality, privacy etc.) • Analyze and visualize information for identifying problems • Foster more direct communication between citizens and policy makers • Translate citizens' input into actionable outputs • Create common understanding across areas of responsibility • Secure buy-‐in from key stakeholders and prevent blocking of new policy by vested interests • Encourage the widespread acceptability of simulation as a public policy tool • Take responsibility for choices made by either him/her or own team.


139 | P a g e

Making a comparison between the ICT tools and methodologies adopted and the challenges and needs of the policy making process, it is clear that detecting and understand problems before they become unsolvable and understanding the actual impact of policies are possible only by the mean of advanced policy modelling and simulation tools and techniques. And as already mentioned, the preciseness of large scale simulations and modelling is allowed and improved by the availability of large amounts of data. From the analysis of these preliminary findings, it seems evident that the use of big data and the way to analyse and exploit them is perceived as an important need by policy makers and it is already in part applied in some sphere of public governance. However, our analysis found that in most cases the application of techniques and methodologies to make sense of massive amounts of data is still in an embryonic stage and it remains largely at experimental level. This is confirmed by the findings of the activity of mapping and identification of case studies that has been conducted as part of the CROSSOVER project, and in part illustrated in the previous section.

4.3. Analysis of the prize winners

The project also launched a prize competition for the best policy-‐making 2.0 application. The prize was assigned based on the criteria of technological innovation, uptake and impact.

Let us present now the description of the three winners stemming from the applications to the prize

IdeaScale SAVE award

Describe briefly the context of the solution

President Obama’s belief was that the best ideas for potential government savings opportunities would come from the front lines (federal employees) In 2009, he launched the SAVE Award (Securing Americans Value and Efficiency), hoping to find ideas that would make government more effective and efficient and ensure taxpayer dollars was spent only on what was necessary. Not only would this help reduce the debt, but it would impact every American tax payer. At that point, there was no existing system that could amalgamate a steady stream of ideas and feedback around those suggestions. Out of desire to remain true to the values of the open government initiative (transparency, participation, and collaboration), the White House selected IdeaScale as the most viable solution that served all of these needs. Not only did it allow employees to submit ideas, but they could vote on those ideas, comment and improve on those ideas and the best ones rose to the top for review. Over the past four years, federal employees have submitted tens of thousands of cost-‐cutting ideas through the SAVE Award. Dozens of the most promising ideas have been included in the President’s Budget. Each year the OMB narrows the best ideas to a “final four.” The American people vote online to choose the winner. The winner then comes to Washington to present their idea to the President. They needed a system that would serve that entire process: submission, voting, evaluation and monitoring, transparent presentation and collaborative development.

What impact did it have on the quality of policies?

Over the past four years, the White House has collected thousands of ideas that cut costs and improve efficiency. This has allowed the White House to meet its main goals:

Main Goals


140 | P a g e

• Generate Suggestions: nearly 100,000 ideas have been collected in the past four years of SAVE Awards. Engagement remains high with thousands of users signing on to submit, vote, and comment each year. These ideas come from every government arena and from numerous geographic locations allowing nationwide collaboration.

• Improve Government Programs and Save Money: Each year a winning idea was selected. Each idea has been assessed as saving the government potentially millions of dollars.

• 2009: As is the case in most hospitals all across the country, medicine that is used in the hospital is not given to patients to be brought home; instead, it is thrown out. Nancy Fichtner proposed ending this practice and sending excess medication home with patients. This is expected to save $21 million by 2014.

• 2010: The winning idea was to reduce the number of hard copies of the Federal Register by offering an opt-‐in choice for those who wanted to be able to access the Register in print. This is expected to save more than $4 million per year.

• 2011: Matthew Ritsko suggested that NASA employees form a lending library of tools that they can share rather than purchasing costly equipment each time they need to build something.

• 2012: Frederick Winter proposed that all Federal employees with transit benefits adopt the reduced senior fare as soon as they are eligible. In the DC area, this change would lower the cost of employee travel by 50% per cent.

• Improve Engagement: Federal employees have stated that they feel more empowered with the tool that is available to them. In its first year, the Executive Office of the President of the United States received 38,000 SAVE Award.

How extensive policy maker and public take up

The application required a minimal commitment on the part of the policymaker, because the ideas had been submitted and prioritized by the crowd at large. Although all ideas were reviewed, the most promising options revealed themselves at an early stage of review. The contribution on the part of each individual was minimal, as well, since the submission of ideas and voting minimized the time commitment for all.

Clear goals and a readymade solution allowed IdeaScale to successfully deploy the SAVE Award community against an extreme timeline. IdeaScale successfully delivered its ideation software on time and within budget to the Executive Office of the President. The platform scaled easily and has never shown any strain under a high volume of users (nearly 90,000 over the past four years of SAVE Awards). In the first week three weeks of 2009 alone, early 40,000 ideas were collected.

Liquid Democracy

Describe briefly the context of the solution

The Enquete-‐Comission Internet and digital Society stands for a parliamentary temporary committee established between 2010-‐2013. During this period, Politicians and experts worked together in order to develop policy recommendations on socially relevant and complex internet policy-‐issues for future purposes. This specific Enquete-‐commission decided – for the first time ever in the German history -‐ to link their decision-‐making processes and to work with an eParticipation platform, which aimed to enable to involved citizens a wider online-‐participation. The platform called enquetebeteiligung.de has been subsequently implemented by the non-‐profit association Liquid


141 | P a g e

Democracy e.V. which is settled in Berlin and which develops further the open source participation software Adhocracy. On this platform involved citizens could build up proposals and solutions for issues concerning the commission, discuss relevant topics and vote upon the proposals. The most popular proposals have been implemented in the final report of the Enquete-‐Comission Internet and digital, the official policy guideline related to the debated topics. The success lies upon that two of twelve recommendations have been mentioned in the final report by taking over exact quotes from the proposals of enquetebeteiligung.de.


For the first time in the German history, citizens could take part through an online-‐process to the work of an official parliamentary committee. Their proposals have been partly included in the final report of the Enquete-‐commission, which is considered as the official policy-‐guideline for the German Government for the next years. Through this process the policy recommendations for the German Government could be provided with a unique democratic legitimation. The fact that the proposals made on enquetebeteiligung.de were of such a high quality that a committee, composed of professional politicians and experts, decided to take them over by mentioning full quotes in their final report. Moreover it proved the opposite to everyone which was the opinion that the average of population is neither interested in legislation nor able to suggest high-‐quality contributions. Enquetebeteiligung.de has shown, that it works. If we strive to enhance the possibilities to involve citizens in online policy-‐making processes, setting this as a democratic goal, we can now have a blue print on how it can be done.

How extensive policy maker and public take up

Enquetebeteiligung.de and the Enquete-‐Commission itself received a wide, positive consideration in the German media. The final report, in which citizen‘s proposals were adopted, has been recently published. According to a scientific evaluation made by the Zeppelin University settled in Friedrichshafen (Germany), the quality of proposals was extraordinary high and the usage of the adhocracy-‐platform for future commissions will be highly recommended. This reflects the high accreditations and comforts the interviewed citizens.

2050 Pathways Calculator


142 | P a g e

Describe briefly the context and functionalities of the solution, and how it was used in policy-‐making UK’s Climate Change Act 2008 set in law a long-‐term greenhouse gas emissions reduction target for the year 2050, as well as a framework for 5-‐yearly “carbon budgets” to reach it. In drafting its Low Carbon Transition Plan in 2009, the first White Paper which sought to bring together the diverse challenges of the newly created Department of Energy and Climate Change (DECC), the Department wished to investigate further what options the country had in meeting its target to reduce greenhouse gas emissions by 80% on 1990 levels by 2050. The Department already had models to understand long term options, such as MarkAl, but none of these was easy or quick to run inside the Department and so senior decision makers did not feel they had an opportunity to interrogate the results. This was the context that pointed to the need for the work, but also highlighted the importance of the ethos of the work – that it should be understandable, radically transparent, interactive, give quick results, and set out easily all the underpinning assumptions. The solution which we developed in DECC’s Strategy Directorate was the “2050 Pathways Calculator”. This is an interactive computer model, available in three formats: the detailed Excel model, a user-‐friendly web tool, and a simplified ‘serious game’ or simulation. https://www.gov.uk/2050-‐pathways-‐analysis By publishing the 2050 Calculator in full, it has enabled a numerate and broader public debate about the UK’s energy demand and supply, and provided a platform which allowed everyone to join the discussion on the same terms. The model seeks to encompass all physically possible outcomes, rather than point to only those thought to be most likely at any one time.


The 2050 Calculator helps everyone engage in the debate and lets Government make sure our planning is consistent with the long-‐term aim. The 2050 Calculator outlines, in minutes, months of work from technical experts. It can be used to engage a range of audiences on the challenges and opportunities of the energy system. It brings energy and emissions data alive, showing the benefits, costs and trade-‐offs of different versions of the future. It allows you to explore the fundamental questions of how the UK can best meet energy needs and reduce emissions. The tool has been shared transparently, both in the sense of sharing all its assumptions and formulations, and also in the sense of sharing its results in a way that people can understand and use.

The analysis has been used in the Government’s Budget statements, Annual Energy Statements and it featured centrally in the UK Government’s Carbon Plan 2011. The team drew out key conclusions from the work which have been picked up by teams across government: for example: the potential doubling of electricity demand over period to 2050 even as energy demand as a whole falls, the limited supply of bioenergy with competing demand in different sectors, its use in understanding the renewable strategy and targets. It has helped senior people in the Department understand issues such as insulation ambition levels, fossil fuel usage, power grid decarbonisation, etc. The 2050 Calculator has been shown to new Ministers when they join the Department. It was used at points such as the Fukushima incident to respond to new questions.

Take up from the public and policy makers

The 2050 Futures team often train colleagues across DECC and other government departments in how to use the 2050 Calculator, and it has been widely used alongside other more detailed models.

Outside of government, the 2050 Calculator has also been widely used. The transparency and accessibility of the approach has led to collaborations from diverse quarters:


143 | P a g e

• With Parliamentary Select Committees and staff in parliament to help give MPs a factual basis for debate.

• With individual enthusiasts and experts who have contributed bug fixes and improvements to our modelling, interfaces and documents.

• With Cardiff University to understand public attitudes to the choices we face when considering the energy system as a whole.

• With the Foreign Office and China, South Korea, Taiwan, Bangladesh, South Africa and the Asian Development Bank -‐ helping each of them to develop their own versions of the 2050 calculator. We are discussing potential partnerships with many other countries.

• With many schools and universities in their teaching. We provided a ‘Schools Toolkit’ to help teachers of Geography, Science, Maths and Citizenship, to use the 2050 Calculator. The Toolkit is most suited to students aged 11 – 16 years old. We also funded a Youth Panel to engage with the work and report to DECC.

• With companies and NGOs, e.g. the infrastructure company National Grid and Friends of the Earth in their own outreach and internal thinking.

In terms of user statistics:

• My2050 has over 16,000 pathways submitted by public. • Team presented to over 500 stakeholders in the autumn 2010 Call for Evidence period • Typically 10,000 unique users of the web tool over a three-‐month period • 100 people registered to use the Wiki (these are the most active Calculator users).

4.4. Lessons learnt from cases and prize

What emerged from the analysis of the prize and the cases is that evidence for uptake is clearly available and now can be considered mature.

However, the evidence presented by the cases and the prize candidates with regard to their impact remains thin and anecdotal in nature. There is no thorough assessment of the impact on the quality of policies. Typically, the impact is demonstrated in terms of:

-‐ Visits to the website and participation rates -‐ feedback and visibility towards media and politicians,

-‐ actual influence over the decisions taken

while the actual impact on the quality of policies is yet to be demonstrated. Some initial work (in the case of Gleam and Pathways 2050) is focussing on comparing the predictions with the reality as it is unfolding. Only the case of Ideascale presents some tangible ex ante estimates of the advantages of the decisions taken through policy-‐making 2.0, but no thourough ex post evaluation.

It is fair to conclude that evidence about the impact remains mostly at the level of actual usage of the final results in the policy decisions. Unfortunately, there is no systematic ex post evaluation of such decisions. This weak evidence base is a major obstacle to encourage further uptake of those solution, and further investment in them. We are far from having robust impact evaluation of policy-‐making 2.0, even at the micro-‐level of individual cases.


144 | P a g e

4.5. An additional research challenge: counterfactual impact evaluation of Policy Making 2.0

The findings of the case studies, the survey and the prize are consistent that no systematic evaluation of the impact of policy-‐making 2.0 is available. This represents a major challenge to further adoption and experimentation in this domain. There is still a number of unresolved questions regarding policy making 2.0 tools and methodologies:

• Do they help engaging new stakeholders and communities? • Do they help predicting impact better than other models? • Do they bring new relevant ideas useful for policy-‐making? • Do they actually lead to better policies?

These questions could be structured in a new evaluation framework for policy-‐making 2.0, which encompassess the full intervention logic, from contextual information, to the intervention, uptake, impact.

Figure 20: a proposed evaluation framework for policy-‐making 2.0

The originality of this model lies in its comprehensiveness, in particularly downstream. Typical evaluation of policy making 2.0 initiatives stop at the level of the level of uptake, such as visitors and users. In the best practices identified, it includes actual influence on the decision taken. The proposed framework includes the actual benefits on the quality of policy making, such as the measurement of the prediction capacity, the improved performance of public sectors, and the improved empowerment of citizens. In this respect, there is a lack of systematic robust evaluation of different policy-‐methods. In fact initial and anecdotal evidence point to the presence of potential impacts, but there is a lack of a proper counterfactual impact evaluation approach available to date. In what follows we present the main methodologies in the field, and how they can be applied to policy making 2.0. We would like to stress the fact that counterfactual impact evaluation is more likely to be used to evaluate policies and initiatives rather than technologies and methodologies. Moreover it is more suitable for evaluating policies impacting a number of distinctive actors.

Evaluating the impact of policies is a complex task because one would like to know what would have been the value for a given output/outcome variable in the absence of the project. This is a value that, by definition, cannot be observed for units not involved in the project. In other words, evaluators cannot know what would have been the behaviour of a treated unit in the absence of treatment. Similarly, we have no counterfactuals for the non-‐treated unit (those not involved in the program). This is a well-‐known problem in policy evaluation analysis (see for instance Neyman, 1923


145 | P a g e

and Rubin, 1974, 1978, 1980, 1986), which has been overcome using several methods. What is common to all these ‘alternative’ approaches is that they attempt to identify or create the most appropriate control group281 in order to overcome the two main obstacles in the estimation of counterfactual

• The 'selection bias', which consists of the fact that target population differs from counterfactual population due to pre-‐intervention features. A solution is the introduction of an identification hypothesis stating that pre-‐intervention variables are sufficient to 'reconstruct' the control group of non-‐beneficiaries (counterfactual)

• The presence of spontaneous dynamics, due to the fact that target population differs from control population for the trend of the result variable. A solution is the introduction of an identification hypothesis to take in consideration the spontaneous dynamics of the result variable trend

There are basically six main counterfactual impact assessment methodologies

Randomised controlled trials

A solution can be found in case of randomized processes (this happens when the possibility to take part to a project is made available to people on the basis of a random process). In this situation we do not expect structural differences between those who are treated (and receive support) and those who are not, so that we can use the non-‐supported subjects as a control group for comparison with the former group.

Difference-‐in-‐Difference (DID)

The impact of a policy on an outcome can be estimated by computing a double difference, one over time (before and after the treatment) and one across subjects (between treated and non treated). This simple method requires only aggregate data on the outcome variable, and at least 3 observations in time: two observations before and 1 observation after. Unfortunately the difference in difference method implies that the trend in treatments and comparisons are the same. With only four points of observation on means we do not know if this assumption is correct. However, with two additional pre-‐intervention data points the parallelism assumption becomes testable.

Regression Discontinuity Design (RDD)

One solution that has been proposed in the literature is the use of so called “regression discontinuity design”. This method can be applied to situations in which it is possible to identify a clear cut-‐off level for treatment access and in which treatment status is based on observable characteristics. In this case the cut-‐off is defined by the eligibility rules of the project so that the treatment group is made up by people that just satisfy these criteria (and hence have access to the project), whereas the control group is composed of people that are just below the cut-‐off level and do not have access to the project. In such a circumstance it is reasonable to assume that the control group and the treated groups are very similar against most criteria, and that the small difference in the variables guaranteeing access to treatment are not sufficient to justify a different value of the outcome variable, so that a difference in the latter can be entirely attributed to treatment.

Instrumental variables and natural experiments

This category is relevant when the exposure to the policy is to a certain degree determined by an external force which does not affect the outcome of the policy directly, but only indirectly, through its influence on the exposure. Angrist and Krueger (2001) define this situation as natural experiment,

281 For an introduction to policy evaluation see Khandker, Koolwal and Samad (2010)


146 | P a g e

i.e. “where the forces of nature or government policy have conspired to produce an environment somewhat akin to a randomized experiment.” There are two main approaches:

• Wald estimator, in which the treatment effect is identified by the ratio of the difference in average outcome between units eligible and not eligible for treatment, weighted by the probability of treatment induced by the instrument. This method is used in case of randomization with partial compliance and randomized encouragement

• Two stage least squares, consisting of a first stage in which is estimated a model predicting the probability of treatment as function of the instrument and other variables, and a second stage in which the outcome equation is estimated using the predicted probability of treatment. This is the case of non-‐randomized natural experiments

Unfortunately this method is not often feasible as does not work when treatment exposure is not mandatory and depends upon some selection process that needs to be controlled for. This is the case at hand, in which the participation to the training projects has been voluntary. Another major weakness of the approach is that it can be difficult to find an instrument that is both relevant and exogenous.

Matching

The most common matching method is the propensity score matching. This approach is based on the premise that, for each unit that has been treated, it is possible to find at least one non-‐treated unit that is “close” enough to the treated counterpart. In this context “close” means that it exhibits a value for the propensity score very similar (if not identical) to the one observed for the treated unit. The propensity score is defined as the conditional probability of receiving the treatment and is usually estimated using logit or probit regressions. After having computed the propensity scores for all the firms in the dataset, it is possible to use this value to match firms in the treated group with at least one firm in the control group. There are various techniques for undertaking this matching process. Some use replacement while others do not, and some use more complex definitions of distance, but the logic in all these approaches is very similar -‐ find a close match for the treated unit within the group of untreated, using the values for the propensity scores. This approach works well if the evaluator has access to a representative sample of the underlying population and can control for all the variables determining the treatment status (the so called “selection on observables” assumption); otherwise the process can be bedevilled with the selection bias issue.

There are three main types of propensity score matching:

• Nearest available matching, according to which each treated unit is matched with the one untreated unit having the most similar initial characteristics

• Radius matching, according to which each treated unit is matched with all of the untreated units having a propensity score within a certain degree of tolerance with respect to the one of the treated unit

• Kernel Matching, in which the outcome of each treated unit is compared with a weighted average of the outcomes of all non-‐treated units

There is a very important difference between propensity score matching and multiple regression analysis. In propensity score matching pre-‐intervention characteristics are different between treated and non-‐treated units, affecting differently the final outcome of the treated and non-‐treated independently from the effect of the programme, thereby creating a selection bias. On the other


147 | P a g e

hand multiple regression analysis makes use of the data from all the treated and non-‐treated units separating the impact on the final outcome due to the different initial characteristic (included in the model as control variables) from the impact of the programme. So the trick is to find as control variables all the initial characteristics that are similar between the treated and non-‐treated units in order to compare the final outcome and interpret the difference as the impact of the programme. When there will be a higher number of eInclusion projects we will adopt also the multiple regression approach in our analysis.

Matching is mostly inspired by outcome additionality and to some extent overlooks behavioural additionality. Findings from matching should always be combined with real-‐time case study evidence to allow some insight into the causality mechanisms. The matched sample approach, in fact, always raises questions of just how similar the subjects are.

Self-‐reported counterfactuals

This approach, employed especially for assessing the issue of behavioural additionality (Aslesen, Broch, Koch, & Solum, 2001; Davenport, Grimes, & Davies, 1998), consist in questioning assisted subjects directly and posing them counterfactual questions. This involves asking the recipients of public support how their employment-‐related behaviour changed, asking formerly supported people how the withdrawal of assistance affected their innovation related behaviour, and asking non-‐supported people how they think their innovation related behaviour would have changed had they received support. Moreover, as one of the objectives of our investigation is to improve the intervention process, the questioning would involve also the intermediary actors. Surveys are a good solution, provided, of course, the respondents do not answer strategically and are able to reflect on behavioural changes in a counter-‐factual situation. The analysis of direct questions on additionality assumes that the respondents are indeed able to reflect on their behaviour in hypothetical, counterfactual situations and that they are telling the truth to the best of their knowledge. However, as respondents have an interest in the continuation of public support, they might be tempted to over-‐emphasize the merits thereof (Sakakibara, 1997). From an opposite perspective, one could argue that some people might be reluctant to admit their dependence on public support. Either way, the differences between hypothetical and real situations should be controlled for through a mixture of matching and self-‐reported counterfactuals.

Challenges and research gaps

• Often we are not facing a natural experiments situation, as the treatment exposure is not mandatory and depends upon some selection process that needs to be controlled

• Often it is not clear which is the treated unit. For example a policy making tool implemented in the internet can affect many groups of people from different countries, and anyway it is very difficult to obtain data on the untreated

• On the other hand often the same units are treated with different policies and initiatives

• Sometimes the treated unit is an entire country: this makes it impossible to apply methodologies such as randomized control trials or matching

• Finally there is the need to develop new sets of indicators used for assessing the impact

The most promising methods seem randomized controlled trials and self-‐reported counterfactuals. Randomized control trials can be used for assessing the impact of policies and initiatives especially at local levels. On the other hand by using the self-‐reported counterfactual method through workshops or survey it would be possible to extract counterfactual information from the agents joining the


148 | P a g e

programs and initiatives, complementing with the investigation of the underlying background information.

Let us see now some examples of counterfactual impact evaluation applied to open government for assessing the validity of claims for transparency and participation:

• Zhang (2012)282 ran a pilot field experiment in Kenya to explore how variation in the content of an information campaign can impact political behavior in villages. The experiment involved two interventions. The first provided a Constituency Development Fund (CDF) report card, which detailed the budgets of all the CDF projects allocated funding in the constituency for that fiscal year, to see if villagers respond to unaccounted for money in locally visible projects. The second intervention, based upon the mixed findings in the literature as to how information can enable citizens to take action couples the report card with a public participation flyer, to see if information about legal rights and decision-‐making processes is necessary for citizens to use the report card to take action

• Olken (2010) ran an experiment in which 49 Indonesian villages were randomly assigned to choose development projects through either direct election-‐based plebiscites or through representative-‐based meetings. In villages where plebiscites were performed, there has been a dramatic increase in satisfaction among villagers, in knowledge about the project, in greater perceived benefits, and a higher reported willingness to contribute. Moreover we have that changing the political mechanism had much smaller effects on the actual projects selected, with some evidence that plebiscites resulted in projects chosen by women being located in poorer areas. According to the outcomes of the study, satisfaction and legitimacy are substantially increased by direct participation.

282 Kelly Zhang, Increasing Citizen Demand for Good Government in Kenya (May 2012) (unpublished manuscript), available at http://cega.berkeley.edu/assets/cega_events/4/Zhang-‐Kelly_Increasing-‐Citizen-‐Demand_Kenya_2012_v2.pdf


149 | P a g e

5. Conclusions: Policy-‐Making 2.0 between hype and reality In this final section, we bring together the findings of the different sections and put policy-‐making 2.0 in perspective of long term improvement of public decision making.

A first question to be addressed is to what extent the research challenges relate to a specific challenge in policy-‐making, as described in section 3 and illustrated in the figure below.

As we can see, the described research challenges capture all the main needs of policy-‐makers, and in particular the capacity to detect problems early and to leverage the collective intelligence in policy-‐making.

Figure 21: Relation Between Policy-‐Making Needs and Research Challenges

In view of this analysis, the next step is to relate each of the research challenges in the policy-‐making cycle. Each research challenge is in fact relevant for one or more of the specific tasks, not for all.

The figure below illustrates this relationship. In each of the phases of the cycle, for each of the tasks, we can identify the potential impact of the research challenges described.


150 | P a g e

The policy cycle starts with the agenda setting phase, where the problem is identified and analysed. In this section, visualization and opinion mining can help to identify the problems at an early stage. Advanced modelling techniques are then used to untangle the casual relationships behind the problem, understanding the causal roots that need to be addressed by policy.

Once the problem is clearly spelled out, we move to the policy design phase, where collaborative solutions are useful to identify the widest range of options, by leveraging collective intelligence. In order to facilitate the choice of the most effective option, immersive simulations support decision-‐makers by taking into account unexpected impacts and relationships. Collaborative governance enables then to develop further and fine-‐tune the most effective option, for example through commentable documents.

Once the option is developed and adopted, we enter into policy implementation. In this phase, it is crucial to ensure awareness, buy-‐in and collaboration from the widest range of stakeholders: social network analysis, crowdsourcing and serious gaming are useful to deliver this.

Already during this implementation, we move into the monitoring and evaluation. Open data allow stakeholders and decision makers to better monitor execution; together with sentiment analysis, they can be used to evaluate the impact of the policy, also through advanced visualization techniques.

In summary, our vision for 2030 embodies a radically different context for policy-‐making 2.0.

On policy modelling and simulation, thanks to standardisation and reusability of models and tools, system thinking and modelling applied to policy impact assessment has become pervasive throughout government activities, and is no longer limited to high-‐profile regulation. Model building and simulation is carried out directly by the responsible civil servants, collaborating with different domain experts and colleagues from other departments. Visual dynamic interfaces allow users to directly manipulate the simulation parameters and the underlying model.

Policy modelling software becomes productized and engineered, and is delivered as-‐a-‐service, through the cloud, bundled with added-‐value services and multidisciplinary support including mathematical, physics, economic, social, policy and domain-‐specific scientific support.


151 | P a g e

Cloud-‐based interoperability standards ensure full reusability and modularity of models across platforms and software.

System policy models are dynamically built, validated and adjusted taking into account massive dataset of heterogeneous data with different degrees of validity, including sensor-‐based structured data and citizens-‐generated unstructured opinions and comments. By integrating top-‐down and bottom-‐up agent based approaches, the models are able to better explain human behaviour and to anticipate possible tipping points and domino effects

On collaborative governance, policy-‐making leverages collective intelligence and collective action. It accounts for the greater policentricity of our governance system. While traditional tools are designed for the public decision-‐makers, these research challenges are more symmetric by nature, in order to engage stakeholders all through the phases of the policy-‐making cycle. Thanks to visualisation and design, it is able to reach out to new stakeholders and lower the barriers to entry in the policy discussions. Policy-‐making 2.0 is not only designed to be more effective, but also more participatory.

This document described at length the specific opportunities of policy-‐making technology, and identified the technological bottlenecks that we need to overcome over the next years if we want to grasp the opportunities of Policy-‐Making 2.0. The research challenges identified so far are not just a simple collection of research issues, but an integrated bundle of innovative solutions that together can lead to a paradigm shift in policy-‐making.

Yet it does not fail us that the main bottlenecks to achieving this vision are not technological. The reason why policy-‐making is not already as open and evidence-‐based as it could be, lies less in the limitation of the technology than on the concrete needs and limitations of human behaviour.

This is a lesson we learnt from many years of studies on the impact of ICT, for example on e-‐government. Regardless of the technological tools at your disposal, the key barriers to change lie with cultural and organisational issues.

We can't claim to propose a more human centric policy making, that takes into account the complexity of human behaviour, and then fail to recognize the humanity of policy-‐makers. Policy-‐makers are agents, and as such are self-‐interested and driven by an own agenda. They are human, and therefore not perfectly rational and atomised. Citizens are human, and not that interested in public policy.

It would therefore be foolish to expect that the simple availability of the technology will suddenly free policy-‐making from politicking, corruption, personal interests or simple incompetence. It is not within the scope of this roadmap to develop generic policy recommendations for improving policy-‐making as such, yet we cannot treat non-‐technological factors as a simple black box: as described in section 2.3, technological tools have to take into account the concrete problems of policy-‐making.

We propose that policy-‐making 2.0 is not a panacea for better government, yet it is not neutral to power relationship that enable such problems as corruption and incompetence to emerge. In other words, these are not “just tools” that can be used for good or bad: they provide the opportunity to re-‐frame the system of check and balances that determine the likelihood of good or bad policy-‐making.

More open data, more transparent models, more visually accountable policy measures can facilitate the uncovering of corruption, personal interests and incompetence. The emphasis on usability and openness of modelling is opening up policy-‐making to a wider range of stakeholders. The availability of different simulated future scenarios enhances the accountability of today’s decision of policy makers.


152 | P a g e

There will always be room for malpractice and greed in policy-‐making 2.0 as well as in any human activities. This is however not an argument to give up on improving the available methods. Raising the barriers to malpractice, and lowering the barrier to good practice, is an achievable goal worth pursuing.


153 | P a g e

6. References 1. Alexopoulos, C. & Kim, S.H. (2002) Output Data Analysis for Simulations, Proceedings of the 2002

Winter Simulation Conference, San Diego (CA), December 8-‐11 2. Alexopoulos, C. & Seila, A.F. (1998) Output Data Analysis, in: Banks, J., ed., Handbook of

Simulation: Principles, Methodology, Advances, Applications and Practice, John Wiley, New York. 3. Balci, O. (1989) How to assess the acceptability and credibility of simulation results, Proceedings

of the 1989 Winter Simulation Conference, Washington (DC), December 4-‐6. 4. Barabasi, A. L., 2003. Linked: How Everything is Connected to Everything Else and What it Means

for Business and Everyday Life, Plume Books. 5. Bederson, B and Shneiderman, B. (2003). The Craft of Information Visualization: Readings and

Reflections 6. Benjamin, A. O. (2010) "Direct Democracy and Local Public Goods: Evidence from a Field

Experiment in Indonesia." American Political Science Review 104.2: 243-‐67. MIT and National Bureau of Economic Research.

7. Bertin, J (1967) Sémiologie Graphique. Les diagrammes, les réseaux, les cartes. 8. Boyd, D., Crawford, K, (2011) Six Provocations for Big Data. A Decade in Internet Time:

Symposium on the Dynamics of the Internet and Society, September 2011. 9. Burkholder, L, ed. (1992) Philosophy and the Computer, Boulder, San Francisco, and Oxford:

Westview Press. 10. Chen, C. (2005). "Top 10 unsolved information visualisation problems." IEEE Computer Graphics and

Applications 25:12-16.

11. Christakis N and Fowler J.“The spread of obesity in a large social network over 32 years,” in New England Journal of Medicine, 357, 370–379, 2007

12. Colecchia, A. and P. Schreyer (2002), ICT Investment and Economic Growth in the 1990s: Is the United States a Unique Case? A Comparative Study of Nine OECD Countries, Review of Economic Dynamics, Vol. 5, No. 2, April, pp. 408–442.

13. European Commission (2009), Impact Assessment Guidelines, SEC (2009) 92 14. Friendly, M. (2008) "Milestones in the history of thematic cartography, statistical graphics, and

data visualization" 15. Gass, S.I. (1983) Decision-‐aiding models: validation, assessment and related issues for policy

analysis, Operations Research, 31(4), pp. 601-‐663

16. Gass, S.I. & Joel., L. (1987) Concepts of model confidence, Computers and Operations Research, 8 (4), pp. 341-‐346.

17. Goldsman, D. & Tokol, G. (2000) Output Analysis procedures for computer simulations, Proceedings of the 2000 Winter Simulation Conference, Orlando (FL), 10-‐13 December 2000.

18. Goldsman, D. & Nelson, B.L. (1998) Comparing systems via simulation, In: Banks, J., ed., Handbook of Simulation: Principles, Methodology, Advances, Applications and Practice, John Wiley, New York

19. Heer, Jeffrey, Fernanda B. Viégas, and Martin Wattenberg. 2007. Voyagers and Voyeurs: Supporting Asynchronous Collaborative Information visualisation. In CHI 2007, April 28–May 3, 2007, San Jose, California, USA

20. Howard, C. (2005), The Policy Cycle: A Model of Post-‐Machiavellian Policy Making?. Australian Journal of Public Administration, 64: 3–13. doi: 10.1111/j.1467-‐8500.2005.00447.x

21. Jorgenson, D. W., Ho, M. S. and K. J. Stiroh (2008), A Retrospective look at the US Productivity Growth Resurgence, Journal of Economic Perspectives, Volume 22, Number 1, 2008, Pages 3-‐24.

22. Keim, D., 2011. Solving Problems with Visual Analytics : Challenges and Applications. Society, 65, p.1.

23. Keim, D. A., Mansmann, F., Schneidewind, J., Thomas, J. and Ziegler H. (2008). "Visual Analytics: Scope and Challenges"


154 | P a g e

24. Kelton, W.D. (1997) Statistical Analysis of Simulation Input, Proceedings of the 1997 Winter Simulation Conference, Atlanta (GA), 7-‐10 December 1997.

25. Kuhlmann, S. & Meyer-‐Krahmer, F., 1994. Practice of Technology Policy Evaluation in Germany: Introduction and Overview. In G. Becher et al., eds. Evaluation of Technology Policy Programmes in Germany. Springer Netherlands, pp. 3-‐29

26. Landesberger, T. von, Goerner, M., Schreck, T. (2009) Visual Analysis of Graphs with Multiple Connected Components. IEEE Symposium on Visual Analytics Science and Technology

27. Landesberger, T. von, Knuth, M., Schreck, T., Kohlhammer, J. (2008) Data Quality Visualization for Multivariate Hierarchic Data, IEEE Information Visualization Conference (INFOVIS), Columbus, OH, USA (2008)

28. Latour, B. (2009). ‘Tarde’s idea of quantification’, in The Social After Gabriel Tarde: Debates and Assessments, ed M. Candea, London: Routledge, pp. 145-‐162.

29. Law, A.M. (1983) Statistical Analysis of Simulation Output Data, Operations Research, 31(6), pp. 983-‐1029.

30. Law, A.M. (2006), Simulation Modelling and Analysis, 4th Ed., McGraw-‐Hill, New York. 31. Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A., Brewer, D.,Christakis, N., Contractor, N.,

Fowler, J.,Gutmann, M., Jebara, T., King, G., Macy, M., Roy, D., & Van Alstyne, M. (2009). ‘Computational Social Science’. Science vol. 323, pp. 721-‐3.

32. Lessig, L. (2009). The New Republic. 33. Livnat, Y. et al. (2005), Visual correlation for situational awareness. IEEE Symposium on

Information Visualization. INFOVIS, 1, p.95-‐102. 34. Nakayama, M.K. (2002) Simulation Output Analysis, Proceedings of the 2002 Winter Simulation

Conference, San Diego (CA), 8-‐11 December 2002. 35. OECD. (2005). Modernising government: the way forward. Paris, France, OECD. 36. Oliner, S. D. and D. Sichel (2000), The Resurgence of Growth in the Late 1990s: Is Information

Technology the Story? Journal of Economic Perspectives, 14: 3-‐22. 37. Ormerod, P., (2010). N Squared: public policy and the power of networks, RSA Pamphlets 38. Pang, B., & Lee, L. (2008). Foundations and Trends® in Information Retrieval, 2(1–2), 1-‐135. doi:

10.1561/1500000011. 39. Phelps ed. (1970), Microeconomic Foundations of Employment and Inflation Theory. New York,

Norton and Co. ISBN 0-‐393-‐09326-‐3 40. Rhyne T.-M. et al.,( 2004) “Can We Determine the Top Unresolved Problems of visualisation?” Proc. IEEE

visualisation, IEEE Press, pp. 563-566. 41. Rosenblum (1994). "Research issues in scientific visualization". In: Computer Graphics and

Applications, IEEE. Volume 14, Issue 2, March 1994 Page(s):61 -‐ 63. 42. Saffo, Paul, (1997), Looking Ahead: Implications Of The Present, Are You Machine Wise. Harvard

Business Review 43. Schelling, T.C., (1969). Models of segregation. The American Economic Review, 59(2), pp.488-‐

493. 44. Sargent, R. (2009). Verification and Validation of Simulation Models. Proceedings of the 2009

Winter Simulation Conference. 45. Schlesinger, M. (1979). Terminology for model credibility. Simulation, 32(3), 103-‐104. 46. Shneiderman, B. (1996) The Eyes Have It: A Task by Data Type Taxonomy for Information

Visualizations, IEEE Symposium on Visual Languages


155 | P a g e

47. Siewiorek, A. Smailagic, J. Furukawa, A. Krause, N. Moraveji, K. Reiger, J. Shaffer, and F. L. Wong, (2003) “SenSay: A Context-‐aware Mobile Phone,” in Proceedings of the 7th IEEE International Symposium on Wearable Computers (ISWC), pp. 248–249

48. Taleb, N. (2008). The Black Swan: The Impact of the Highly Improbable. Penguin. 49. Thomas and Ahrweiler Eds. (2005). Internationale partizipatorische Kommunikationspolitik -‐

Strukturen und Visionen. Münster: LIT. 50. Thomas, J.J. and K. Cook (2003), Illuminating the Path: The R&D Agenda for Visual Analytics.

2005: IEEE 51. Tizzoni M., M., Bajardi, P., Poletto, C., Ramasco, J., Balcan, D., Gonçalves, B., Perra, N., Colizza, V.,

Vespignani, A., (2012). Real-time numerical forecast of global epidemic spreading: case study of 2009 A/H1N1pdm, BMC Medicine, 10:165.

52. Tufte, E. R (1983) The Visual Display of Quantitative Information, Graphics Press 53. van Wijk. (2005). The Value of Visualization. In IEEE Visualization Conference. pp. 79-‐86 54. Ware, C. (2004) Information Visualization -‐ Perception for Design, Second Edition, Academic

Press 55. Ware, Colin (2008): Visual Thinking: for Design. Morgan Kaufmann


156 | P a g e

56. Wattenberg, M. (2008) The Word Tree, an Interactive Visual Concordance, IEEE Transactions on Visualization and Computer Graphics, Vol. 14, No. 6


157 | P a g e

7. List of Acronyms ABM: Agent Based Models

ADAMS: Anomaly Detection at Multiple Scales

APES: Agricultural Production and Externalities Simulator

BI: Business Intelligence

BRICs: Brasil, Russia, India and China

CAPTCHA: Completely Automated Public Turing test to tell Computers and Humans Apart

CDC: Center for Disease Control and Prevention

CGE: Computational General Equilibrium Models

CINDER: Cyber-‐Insider Threat

COMA: COllaborative Modelling Architecture

CSCW: Computer-‐Supported Cooperative Work

CTR: Click-‐through Rate

CVADA: Center of Excellence on Visualization and Data Analytics

DARPA: Defense Advanced Research Project Agency

DBMS: Database Management Systems

DECC: Department of Energy and Climate Change

DID: Difference-‐in-‐Difference

DHS: Department for Homeland Security

DOE: Design Of Experiment

DSGE: Dynamic Stochastic General Equilibrium Models

DW: Data Warehouse

ECB: European Central Bank

EDW: Enterprise Data Warehouse

EEA: European Economic Area

eID: Electronic Identity

ERP: Enterprise Resource Planning

ETL: Extract, Transform, Load

FAO: Food and Agriculture Organization

GHG: Greenhouse Gas

GIS: Geographic Information System

GLEAM: Global Epidemic and Mobility Model

GPS: Global Positioning System

GSS: Global Systems Science


158 | P a g e

HCI: Human-‐Computer Interaction

HLA: Higher Level Architecture

ICT: Information and Communication Technologies

ILEs: Interactive Learning Environments

I/O: Input/Output

KPIs: Key Performance Indicators

LOD: Linked Open Data

LMS: Learning Management Systems

LTA: Land Transport Authority

MarkAl: MARKet ALlocation

MAS: Multi-‐Agent Systems

MPOs: Members of Public Organizations

MPP: Massive parallel processing

NAF: New America Foundation

NASA: National Aeronautics and Space Agency

OECD: Organisation for Economic Co-‐operation and Development

OGD: Open Government Data

OGPL: Open Government Platform

OLAP: On-‐line analytical processing

OOP: Object Oriented Programming

PMOD: Policy Modelling

P2P: Peer-‐to-‐Peer

RDD: Regression Discontinuity Design

RDF: Resource Description Framework

RTAP: Real-‐time analytics processing

SaaS: Software as a Service

SAML: Security Assertion Markup Language

SMD: Social Media Data

SPARQL: SPARQL Protocol and RDF Query Language

STREP: Specific Targeted Research Projects

TRM: Technology road-‐mapping

T21: Treshold 21

UNOSAT: United Nations Operational Satellite Applications Programme

XML: eXtensible Markup Language


159 | P a g e


160 | P a g e

0205 f01 international research roadmap

Technology

Transcript of 0205 f01 international research roadmap