Overall Evaluation of the City.Risks...

Deliverable D6.3

Overall Evaluation of the City.Risks Approach

Editor T. Paulin (SYNYO)

Contributors E. Wepsäläinen (MLM), B. Bhardwa (ICPR), T. Paulin (SYNYO), S. Costicoglou (SPACE)

Version 1.0

Date April 30th, 2018

Distribution PUBLIC (PU)

Executive Summary

This deliverable provides an overview of the piloting activities executed in the

City.Risks project, covering the period July 1st 2017 until April 30th, 2018. Building

upon D6.1 - The Detailed Pilot Plan - and accompanying D6.2 - The social aspects of

the pilots, this document provides an overall summary of what has been done and

emphasises the lessons learned from the piloting activities and efforts that have been

invested into the process.

Table of Contents

1. INTRODUCTION .......................................................................................... 4

1.1. Project and Work Package Scope ....................................................................... 4

1.2. Overview and outline .......................................................................................... 4

2. PILOTING ACTIVITIES OVERVIEW ........................................................................ 6

2.1. Data Collection Methodology ............................................................................. 6

2.2. Planned Long-Term Piloting Activities ................................................................ 7

2.2.1. Target Group in Sofia .............................................................................................. 8

2.2.2. Target Group in Waltham Forest ............................................................................ 8

2.2.3. Target Group in Rome ............................................................................................ 9

2.3. Planned Simulations............................................................................................ 9

3. ALTERATION MADE TO THE ORIGINAL USE-CASES ............................................... 10

3.1. Simulation Activities Use-cases......................................................................... 10

3.2. Long-term Pilots Use-cases ............................................................................... 10

3.3. Operational Centre Usefulness Tests ................................................................ 11

4. COMMUNICATION CHANNELS AND DISSEMINATION MATERIALS ............................. 12

4.1. Communication Channels ................................................................................. 12

4.2. Dissemination Materials ................................................................................... 13

4.2.1. Project website dedicated pilot pages.................................................................. 14

4.2.2. User Guide for the Mobile Application ................................................................. 15

4.2.3. Flyer ...................................................................................................................... 17

4.3. General Feedback to Dissemination Materials ................................................. 19

4.4. Dissemination Materials in Rome ..................................................................... 19

5. PILOT SITE SPECIFIC RESULTS .......................................................................... 22

5.1. Waltham Forest ................................................................................................ 22

5.1.1. Simulations ........................................................................................................... 22

5.1.2. Long-term Pilots - Recruitment efforts in Waltham Forest .................................. 22

5.2. Rome ................................................................................................................. 24

5.2.1. Simulation ............................................................................................................. 25

5.3. Sofia................................................................................................................... 28

5.3.1. Simulation activities .............................................................................................. 28

5.3.2. Feedback on Simulations and Long-term Pilot ..................................................... 30

6. PILOT ACTIVITY MONITORING ........................................................................ 31

6.1. Demographics, affinity and geographical location ........................................... 31

6.2. Gender Perspective ........................................................................................... 32

6.3. User Activity ...................................................................................................... 33

6.4. User-activity Reported by the Roma Capitale Social Media Pages. ................. 34

6.5. Analysis of Firebase data .................................................................................. 34

6.6. Geographical Distribution of Survey Participants ............................................. 36

6.7. Issue tracking .................................................................................................... 37

7. CONCLUSION .............................................................................................. 38

8. ABBREVIATIONS .......................................................................................... 40

9. FINAL VERSION OF QUESTIONS AND QUESTIONNAIRES ......................................... 41

9.1. Focus Group Discussion Guide - Second Version .............................................. 41

9.2. Final versions of the surveys ............................................................................. 42

9.2.1. City.Risks Survey 1 ................................................................................................ 42



1. Introduction

This report provides an overall overview and summary of the piloting activities performed as part of the City.Risks project in order to evaluate the City.Risks platform. The report’s primary focus is on documenting the playout of the events and activities that were performed during the pilot, as well as the main observations.

Overall, the piloting activities consisted to two main pillars; long-term pilots and simulation activities, across three pilot sites in different locations across Europe; Sofia, Rome and Waltham Forest. The long-term pilots intended to have a strong user focus, where users had the City.Risks mobile application and its functionality available for a certain duration of time – e.g., one months, while their usage of the mobile application was monitored and their feedback collected. To support the long-term pilots, a series of focused simulations where selected use-cases were set up and executed with volunteers as to validate the functionality of the system and to gather overall impression about the functionalities that have been developed as part of the project.

1.1. Project and Work Package Scope

WP6 is responsible for piloting the development efforts in the other work packages of the City.Risks project. This includes planning, execution, monitoring and the evaluation and processing of piloting activities, surveys and other user inputs. D6.1 details the planning and necessary resources to execute the pilots, namely the overall time plan, the methodology to be used in the pilots and materials such as informed consent and participant information sheets that would be necessary throughout. D6.2 reports on the evaluation of the platform from a user perspective and the analysis of the collected user-inputs; focus group discussions, surveys and feedback through other communication channels. Finally, D6.3 (this document), provides an overall documentation and evaluation of the piloting activities that were done throughout the last phase of the project.

1.2. Overview and outline

This report is divided into five main sections, covering the different piloting activities and outcomes: The Piloting Activities Overview section summarizes what was planned for the piloting activities, and the overall data collection methodology, as well as mapping out how these differed from original pilot plan (documented in D6.1) as it was defined in July, 2017. The Alternations Made to the Original Use-cases section provides a revised version of the original use-cases and scenarios to which the participants were exposed, reflecting over challenges in communicating these use-cases to participants. The Communication Channels and Dissemination Materials section documents how communication with active and potential participants was facilitated as well as a overview of the communication materials that were developed to support the communication. Pilot Site Specific Results delves deeper into the activities at each pilot site and summarizes what was done. Finally, Pilot activity

Monitoring provides technical details regarding the pilots, across all pilot sites, documenting various user-activities and demographics results.

2. Piloting activities overview

In this section, we provide an overview of the planned piloting activities as they were outline before the beginning of the piloting phase - outlined in detail in D6.1 - summarizing the overall structure of the pilots regarding data collection, evaluation and methodology. Throughout the planning phase, the main focus was on scoping the longer-term pilots and to define a suitable environment in which the City.Risks platform could be evaluated by combining various tools and mechanisms as elaborated on below.

Figure 1: Pilot sites and their planned long-term pilot activities

The long-term pilots were scheduled to run for one month each, starting October and ending end of December, across the three pilot sites as shown in Figure 1 (Above), which shows the expected months for the piloting activities in each city. At each site, four main activities, namely a short-term simulation, to kick-off the pilots at each site and ensure that everything is working correctly, the long-term pilot with users from the local areas, and finally a focus group discussion, to round of the pilots and to collect feedback for participants that showed interested.

2.1. Data Collection Methodology

The overarching pilot-plan consisted of connecting the main components as outlined in Figure 2. The overall concept was to interlink each of the activities, i.e., the three long-term pilot surveys, the user activities recorded through the mobile application and finally the opt-in focus group discussion with interested participants that would like to engage in further discussions about the City.Risks platform and ecosystem.

Figure 2: Key activities planned for the piloting phase

Figure 3: Data collection and data linkage

The evaluation methodology was implemented in such a way that the various activities could be interlinked with a unique identifier (UID), thus connecting the answers in the three (two of which – the first and last – the user had to participate in, in order to be eligible to be included in the prize draw, which was used as an incentive for the users to participate). The surveys were designed in such a way that essential information, such as demography, fear of crime etc, would be collected already in the first surveys, such that valuable information would have been collected early on, but any personalised information, would be only collected as part of the final survey, and only on a voluntary basis, if and only if the user wished to participate in either the prize draw and/or in the focus group discussion, since the information would have be necessary to contact them directly with an invitation. Finally, the UID would allow the correlation between the user’s level of activity (without revealing what the user did exactly) during the piloting activity, with their survey feedback about the City.Risks platform, as their overall level of engagement could help to make more detailed conclusions based on their answers.

2.2. Planned Long-Term Piloting Activities

For the long-term pilots, the plan was to evaluate all City.Risks use-cases, except UC1 and UC2. These two use-cases were excluded for practical and conceptual reasons, for example, due to the limited number of Bluetooth sensors and a rather strong focus on integration, respectively. The remaining use-cases, could be evaluated independently, rather it was decided to present the use-cases as a collection of community tools that support each other, where the users are able to interact directly with their community (UC3) correlated in time and space, or with their local authorities, through for example UC4. The remaining use-cases would provide accompanying functionality such as routing and mapping of various criminal activities in various geographical locations.

Figure 4: Planned long-term piloting activities

During the preparation, the primary discussion was on defining and preparing specific practical details such as a suitable target group(s) for the long-term pilot, as elaborated upon below, the suitable duration of the pilots at each pilot site, how to ensure the retention of the participating users through incentives and how many users would be required to validate the platform. The main strategy is summarised in the above figure, which was also presented during the second review meeting.

2.2.1. Target Group in Sofia

The target group in Sofia was security guards working at VIP Security, then G4S, as the core group using the mobile application as a tool and generating content during the pilot in the test area, and external participants, i.e., citizens, contributing with their feedback to said content. The company is one of the Consortium Partners and it was therefore easy to access this first group. The idea was that the security guards would use different features of the City.Risks app in the course of their daily work activities, dependent upon the various situations they may encounter.

2.2.2. Target Group in Waltham Forest

In London, Waltham Forest, the target group for the pilot was several different Neighbourhood watch groups that were to be invited via social media, e.g. via Facebook groups. These groups were deemed appropriate participants as they had an interest in crime and safety. Their Facebook groups also had many members providing a great opportunity to reach out too many potential users at the same time. A contact person at the Waltham Forest Safer Neighbourhood Board was asked to distribute information about the pilot project via social media platforms. Targeted groups included for example “Leytonstone Life” (11,000 members), “Walthamstow Life” (17,000 members) and “Crime Awareness Waltham Forest” (2,300 members). The distribution of information regarding the City.Risks app.

2.2.3. Target Group in Rome

The main target group for the pilot in Rome were cyclists which in urban areas like Rome, continue to remain the main thefts target. During the planning activities, it was discussed that the partners from Rome would have access to a large organisation of cyclists, which would be a highly suitable target audience for the City.Risks pilots, due to the applicability of UC1. The main objective was to engage this community of citizens to collaborate closer with Roma Capitale Local Police to report suspicious activities and provide a hands-on tracking during the short-term pilots.

2.3. Planned Simulations

Three use-cases were selected to be part of the evaluation, namely UC1 - Theft of Personal Belongings, UC3 - Request for Witness - and UC4 - Gather and Disseminate Information for Ongoing Events. These could be exposed to functional testing though the simulations, under real-world conditions without much modification in the explanation of the use-case to the users.

Figure 5: Planned simulation activities

For convenience, the individual use-cases can be mixed and combined, for example, UC1 and UC3, such that a broader range of users can be included; both executing the use-case in real-time, actively searching for the Bluetooth tags while traversing the testing area, as well as retrospectively, where other participants may have observed something. Similarly, UC3 and UC4 can be combined into a tool where users report an ongoing activity by utilizing UC4 functionality, and operators can use UC3 to seek out potential witnesses to gather additional information.

3. Alteration Made to the Original Use-cases

For the piloting activities, a subset of all City.Risks use-cases were selected for both the long-term pilot and the simulation activities, as outlined in the previous section. In this section, the use-cases are elaborated upon, in order to highlight the differences between the originally defined use-cases and how they were presented to the potential participants. This is the case in regard to the long-term piloting use-cases, as they needed to be reformulated in order to be evaluated by external users.

3.1. Simulation Activities Use-cases

UC1, UC3, UC4 were selected as being most suitable for the short-term evaluations with users, since they were the most meaningful to evaluate under a simulated scenario, where the parameters could easily be defined and controlled and the participants instructed in detail what to do. E.g., it could be ensured that the penetration rate of mobile devices with the City.Risks app equipped was high enough as to be sure to encounter the Bluetooth tags, or that multiple City.Risks users would be able to collaboratively report on the same ongoing event. Future, whenever possible, the scenarios were compiled into one ‘story’. For example, UC1 and UC3 can be executed as one, supporting use-case, where the user can ask the City.Risks platform to search for their stolen tag automatically, through the Bluetooth sensing functionality, as well as request a witness in addition, thus making the scenarios increasingly interactive as well as enabling participation from users without Bluetooth integrated into their mobile devices.

3.2. Long-term Pilots Use-cases

For the long-term pilots, UC3, UC4, and UC5 were selected. As a result of the limited number of Bluetooth tags available, UC1 could not easily be made available, also this use-case did not fit naturally in an organic execution of a long-term pilot. At the same time, the use-cases that were selected for the long-term pilot were slightly renamed, as to remove any negative connotation in their original wording (E.g., Guide me via a safe route was changed to Guide me via an alternative route). This was done in order to make them more appealing to the participants, and to remove the need to have something unfortunate happen to the participant before the City.Risks mobile application became of use. The use-cases were defined as follows:

- UC3 - Request for Witness - was promoted as a community building tool, where users could send out various requests to their community and interact with them directly through the built-in chat functionality.

- UC4 - Gather and Disseminate Information for Ongoing Events - was presented as a general reporting tool, where participants could, in real-time, report various events and concerns they encountered in the course of their daily routines. In the second attempt to restart the pilots in Waltham Forest, the use-case was change to make it a dissemination tools for authorities, where

e.g., the city council could push information about on-going events, organised by the council, to citizens, restricted by a geographical area of relevance.

- UC5 was made available during the pilot in order to provide as many features as possible to the users. However, this use-case was given less focus since it depends on events occurring, which the user could then be routed around.

3.3. Operational Centre Usefulness Tests

The Operational Centre evaluation was prepared as part of the piloting activities and was to run in parallel to these other tasks. Since it was not feasible to have an operator actively moderating the Operational Centre over the full duration of the pilot, the main purpose was instead on collect focused feedback from the operators regarding how useful they found the City.Risks platform and how it could be helpful in their daily activities. The prepared scenario consisted of a combination of UC1, UC3 and 4, where the operators would have been shown the functionality of the of the City.Risks platform though a general overview and exemplified through the use-cases. The combination of the three use-case allowed for a comprehensive overview, where e.g., a user would register their device stolen through the in-app functionality, thus the event being reported at the operational centre. The operator could then use the system to track the movement of the stolen Bluetooth tag, and use UC3 to request other witnesses that may have observed something, while at the same time informing the general public that a theft had occurred by broadcasting a message using UC4.

4. Communication Channels and Dissemination Materials

As part of the planning phase and the piloting activities, the consortium setup various communication channels, to enable communication to and from the participants. Similarly, various dissemination materials were generated iteratively as to provide the potential participants with an overview of the piloting activity.

4.1. Communication Channels

Interaction between the project consortium, (potential) pilot participants, the responsible authorities in each pilot site and the external parties that were involved in the execution and coordination of the piloting activities, was facilitated by various means, as described in this section.

Project website - The project website1 served as a tool for communication of the project as a whole, providing general information about the project purpose and use-cases, the consortium as a whole and project funding from the European Commission. The content was translated into two languages; English and Italian. The page served as support material for the interested participants since it provided information about the project as a whole.

Dedicated pilot information web pages - A dedicated webpage, one for each of the three pilot sites, was created to provide a detailed overview of the pilot specific use-cases - modified versions of the original City.Risks use-cases - that the users were expected to utilise during the piloting activities, the timeline of the pilot, what the user would get out of the pilot, i.e., participating in a prize draw, and their overall right, as elaborated in the participant information sheet.

Local Contact - Each pilot site had a local contact whose task was to facilitate the pilots and help to recruit and motivate potential ‘candidates’ for the pilot within their organisation, community or local area. Each pilot site also had a dedicated contact email that could be used to communicate with both potential and active pilot participants. This was widely disseminated locally and embedded into each of the three versions of the localised mobile application.

Emails and Mailing lists - The primary communication with both potential and actual pilot participants, was facilitated through email communication. Namely, the users were provided with a site-specific email, as listed in Table PPP, as well as the local contacts at each site used their network, which largely consisted of mailing lists.

1 http://project.cityrisks.eu

Table 1Table PPP: Dedicated piloting support emails

Pilot Site Email

[email protected] London, Waltham Forest

[email protected] Sofia

[email protected] Rome

Social Media - The social media channels of both the project itself, i.e., the official project Twitter account, as well the Facebook and Twitter accounts were used to disseminate information to potential participants for the piloting activities. This was in particular the case in Rome where their official accounts were used to disseminate the information.

In-app Communication - Besides the functional interaction with the City.Risks mobile app, the mobile app provided a direct communication channel through a dedicated email, one for each of the three pilot sites, through which the users could contact the team directly. This feedback channel was conveniently integrated into the mobile application as to make it easy for the participants to send an email directly from the mobile application.

Surveys - One of the main communication channels for collecting feedback were the surveys. Three identical surveys per pilot site provided the means through which the users could provide their feedback regarding three key themes: general perceptions of safety; safety in the context of various demographics, one serving as a gentle reminder of the City.Risks application; and finally, an application specific survey of perceptions of safety as well as feedback on various aspects of the mobile app itself.

4.2. Dissemination Materials

In order to inform and attract participants to the piloting activities, various dissemination materials were created to explain the purpose of the pilots and what participants should expect. Among others, these materials included the participant information sheet that was developed during the planning activities in D6.1, to ensure that the participants were aware of what their rights were. Besides the general, existing project materials and information the following materials were developed as part of the piloting activities:

- A dedicated pilot webpage for each pilot site, translated to the local language and containing:

- Participant information sheet - How to register - User manual - Access to downloading the City.Risks mobile app

- Flyers, providing a high-level overview of the pilots. - Presentations detailing the pilot simulations and the scenario executions.

4.2.1. Project website dedicated pilot pages

As part of the preparation for the long-term-term pilots, a series of user targeted overview pages were developed, localised to the pilot sites; Waltham Forest, Sofia and Rome. These info-pages contained:

- The participant information sheet, outlining the rights of the participant. These sheets had been developed as part of D6.1 but were developed to be more dynamic and interactive on the project website.

- An overview of the use-cases that participants should expect from the mobile application, see Section 4.2.2 for details, as well as user-guide for each.

- A direct link to the installable application that was hosted on the project website. Unfortunately, this led to users having to go through an extra step during the installation process, since the application was not provided through an official app store, and the users needed to accept the installation of application from third parties.

Figure 6: Landing page of the English version of the pilot presentation

Figure 7: Details about the pilot activities in Waltham Forest, reflecting the participant information sheet.

4.2.2. User Guide for the Mobile Application

To help the users to get and use the mobile application, a series of visual user guides were developed, covering how to install the mobile application in the first place, and one for each of the applicable use-cases. E.g., UC1, UC3 and UC4 for the simulations and UC3 and UC4 for the long-term pilots2.

Figure 8: Installation guidelines

2 During the long-term pilot, UC5 was available on the mobile application as well, but was given less

attention since in majority of the cases it provides ordinary routing functionality, and it would not be feasible that an event would occur that the users would actually need to be routed around. Rather UC5 was provided as part of the whole City.Risks platform.

The main steps for reporting a tag stolen in UC1 are illustrated in Figure 89. Whenever a user reports the tag stolen, the event is communicated to the operational centre and it's up to the operator to determine the best course of action while the device is being tracked through the gateways and other City.Risks mobile app users.

Figure 9: User guide for UC1 - Reporting the tag stolen

Figure 910 illustrates the steps needed to create a request for witness, though the City.Risks mobile application. To add a request, the user simply has to provide a short description of what occurred, when it happened and specify the location of where it happened. Optionally, the user may provide a photo of the event or item, as to provide a visual reference to the potential witnesses.

Figure 10: User guide for UC3 (Top) and UC4 (Bottom)

4.2.3. Flyer

For Waltham Forest a flyer was outlined, reflecting the official logo as well as a reference to the funding of the project, as to give the information additional authority. The flyer provided information about what the user should be able to expected, when the piloting activity was to be executed, and especially what was in for the potential participants. Further, direct links to pilot website, where the users may find additional information as well as contact information was added such that any questions may be forwarded to the consortium.

Figure 11: Flyer for dissemination the pilot activity in Waltham Forest to attract users.

4.3. General Feedback to Dissemination Materials

One of the largest challenges when designing the dissemination materials was achieving a balance between providing meaningful use-cases for participants during the long-term pilots while at the same time creating distance from the core use-cases that had been developed in order to shape the City.Risks platform as a whole. The core City.Risks use-cases, while useful in their own way, have negative connotations, and are primarily intended to help people after an unfortunate and often criminal event. For the pilots, these use-cases needed to be altered in such a way as to evaluate the relevant scenarios without the participants having to actually experience an unfortunate event. For this reason, the overall piloting concept was described in general terms. Unfortunately, this resulted in a lack of clarity among users (as described in D6.2) about the aim of the application and how it might be used. To alleviate this, the second attempt in Waltham Forest used UC4 as a tool for disseminating interesting event by e.g., the city council, so that users would be notified if something was being organised in their vicinity. Further, more detailed documentation of the communication activities is documented in the pilot site specific discussion in the following sections.

4.4. Dissemination Materials in Rome

In Rome, additional dissemination materials were created based on the existing project dissemination materials as well the materials described above. These were posted on the official communication channels of Roma Capitale, namely the official website, the Twitter channel, and Facebook. These are provided below.

Figure 12: Dissemination of City.Risks on the official website of Roma Capitale

Figure 13: Dissemination materials on the official Twitter of Roma Capitale

Figure 14: Dissemination on the official Facebook page of Roma Capitale

5. Pilot Site Specific Results

Figure 15, outlines the main piloting activities: the short-term simulation, correlated with consortium meetings at the three pilot sites, coordinated internal testing of the main piloted components, and the execution and duration of the long-term pilots. Further, since the long-term activities in Waltham Forest were restarted three time in order to increase attendance, a case study on the recruitment efforts is provided here as well, besides reporting on the activities.

Figure 15: Timeline of key activities during the piloting phase

5.1. Waltham Forest

Waltham Forest was the first pilot site to kick-off the piloting activities, and therefore played the role of piloting the pilots. In Waltham Forest, the piloting activities were officially kicked-off during the seventh general assembly consortium meeting in London, organised by ICPR and WF. This was a two-day event, where consortium partners prepared the various systems necessary for the simulations that were scheduled for the next day, with the intention of starting the long-term pilot the week after. Unfortunately, technical issues arose during the simulations that resulted in the postponing of the long-term pilot activities. Overall, three attempts were made to kick-off the long-term pilots, namely as initially planned, followed up with a secondary attempt on a long-term pilot. And finally, a rescoped approach, as elaborated below.

5.1.1. Simulations

The simulation activities were executed at the Waltham Forest Town Hall, where the use-cases were tested. The main focus was on the technical aspects of the City.Risks platform and on getting the participant familiarised with the mobile application, explaining the functionality and discussing alternative use-cases and areas where the technology could be applied. For example, suggestions came up where the Bluetooth sensor could be installed on pets, and then be traced in the event that they run away or are stolen, or other ways where the reporting for both UC3 and UC4, could be used when interacting with one's own community as well as with authorities.

5.1.2. Long-term Pilots - Recruitment efforts in Waltham Forest

It was initially planned that our pilot site partner would recruit around 100 participants via the existing Neighbourhood Watch Programme to take part in the long-term pilot

activities. This is a group with a vested interest in urban safety and security. The Neighbourhood Watch Programme consists of three target groups: “Leytonstone Life” (11,000 members), “Walthamstow Life” (17,000 members) and “Crime Awareness Waltham Forest” (2,300 members). These geographically-spread target groups have an active Facebook presence and were going to be approached via social media to take part in the City.Risks pilot trials.

In addition, as a London Borough, our pilot partners in Waltham Forest are also well-connected with other community groups (e.g. a Women’s Group set up to deal with specific localised issues that they were facing) and council-wide mailing lists and networks. Amongst these networks were key individuals, such as the Chair of the Waltham Forest Safer Neighbourhood Board, with a stake in safer communities.

These various channels were used to recruit participants for the long-term trials. Internally, our pilot partner was in contact with the Communications Team to increase publicity about the project and the piloting activities specifically. Our pilot partner also handed out project flyers in an attempt to boost recruitment.

In addition, to increase recruitment in Waltham Forest, ICPR submitted an ethics amendment to Birkbeck, University of London’s Ethics Committee.

When the ethics application was initially submitted in July 2017, the research team at ICPR had stated:

“The researchers at The Institute for Criminal Policy Research (ICPR), Birkbeck are not involved in the recruitment of research participants. The recruitment of research participants will be administered locally by the City.Risks consortium partners in the three pilot cities: London Borough of Waltham Forest (LBWF) in United Kingdom; Sofia in Bulgaria and Rome in Italy.”

However, by October 2017, ICPR submitted an amendment to assist the London Borough of Waltham Forest with the recruitment of research participants for the long-term pilot trials. It was stated that this would involve contacting the Communications Team at Waltham Forest Council directly and asking them to disseminate recruitment material on the project’s behalf; along with assisting colleagues at Waltham Forest with recruitment via the local college in the area.

Externally, the Chair of the Waltham Forest Safer Neighbourhood Board was a keen supporter of the City.Risks app and described himself in email correspondence as our “critical friend”, providing feedback throughout the lifetime of the piloting activities. His feedback included comments on a number of drafts of the recruitment flyer we disseminated; critiquing the ‘academic’ language used in earlier drafts which he thought would impede recruitment. In his role as Chair, his assistance and reach with the recruitment efforts was pivotal. However, whilst holding the strategic position to assist with the recruitment drive, his initial reticence about the pilots was shared early on, expressing concerns regarding that the purpose of the application could be improved and communicated better, and would wish for more specific scenarios for how to use the application. After some clarifications, especially focusing on that the purpose was to see how people would use the application, given the functionality that it provided, the information was dissemination within his network.

Another concern about the application was also echoed by a Facebook user who emphasised the concept is quite interesting and a great idea, but at the same time raised concerns about unintentional usages of the mobile app-. These concerns included the risk of vigilantes taking matter into their own hands or that, people would flock to incidents to take photos or, contrary to the purpose of the application, participate in the activities. The same users also commented that the mobile application should be endorsed by an authority, as to improve transparency regarding where their data that the users submit end up at.

Overall, throughout the recruitment process, the following concerns were raised:

● Usability issues with the mobile app

● Clearer instructions on how to use the app

● The App should be endorsed by the relevant authorities

● Transparency about what happens to uploaded data

● Concerns about unintended app usages

After taking the feedback into account, a third attempt was made, in which the pilot scope was changed to last a week, targeting 10 participants and all would be incentivised through vouchers, as well as creating updated materials that were simplified. However, it was not possible to muster enough people to execute the trial.

Since a broad long-term pilot activity was not feasible due to limited participation, a Citizen Consultation was also proposed as late on as February 2018 following a Consortium GA meeting in Sofia in a final recruitment push. It was suggested that an informal approach was needed which required participants to use the City.Risks app whilst they attended the meeting and to provide feedback whilst they were there and it was envisaged that the App/its concept would stimulate a quasi-focus group discussion. The Consultation was then changed to an email exchange, in which some of the contacted participants provided feedback as is outlined in deliverable D6.2.

5.2. Rome

In Rome the city administration was the primary coordinator of the pilots, and

invested efforts into the dissemination, planning and execution of different activities

across the duration of the piloting phase. Here, a significant effort was put into the

dissemination of the pilot activities as well, where the official channels of the Roma

Capitale were taken into action. The below lists the main activities in Rome.

• Operative activities to prepare the long-term and short-term pilot actions started on 12/10/2017.

• On 13/11/2017 publication on official web, Twitter and Facebook of posts (informing on the event on 2nd of December.

• 45 citizens sent an email to take part in the event on 2nd of December.

• 11 citizens took part in the 2nd of December event and filled the 1st survey questionnaire. Another seven citizens sent by email the 1st Survey

questionnaire filled. The total number of citizens that took part in the 1st Survey were 18.

• The 1st Survey was fully completed on 21st of December sending the questionnaires filled in PDF to the Consortium.

• The 2nd survey activities started on 14/12/2017 and they were completed on 26/12/2017. 8 citizens completed the 2nd survey.

The 3rd survey activities started on 19/01/2018 and they were completed on 27/03/2018. 8 citizens completed the 3rd survey. The experience and data collected through the pilot in Rome, have been considered by Commander of Local Police Mr Diego Porta as very important and they will be considered seriously to be incorporated within the SIRS: Safe Rome Integrated System, which is, at present, managed by Rome Local Police, and will surely give a precious support to improve Local Police and Police Forces operational working methods to prevent and fight against crime and to help the municipal administration decision-making processes concerning urban security.

Not by chance Italian legislative Authorities in the very recent DL 20th February 2017, n. 14 signed with modification into the law n. 48 dated 18th April 2017 containing “Urgent dispositions for security in towns”, have demonstrated to have real faith in the potentialities of the project where they have established, with two different law dispositions, the possibility to “involve” voluntary organizations, professional associations and however called committees to reach the goal of urban security and to fight against urban decay.

In the art. 5 of this law has been established the possibility to engage, “by the means of appropriate agreements, territorial nets of volunteers to protect and safeguard the street furniture, green spaces and city parks.

The DL n. 14/2017, at the art. 7, subparagraph 1 bis, says that “In order to achieve a better spread of urban security actions in the territory, and for further public interests”, …. the abovementioned agreements “can concern projects proposed by housing managing bodies or by building superintendents, associations, any kind of committees of companies, professionals or residents to install, at their own expenses, technologically advanced monitoring systems, equipped with video analysis software for the active monitoring and the automatic alarm to police forces stations…”

5.2.1. Simulation

The piloting simulation was undertaken in two sessions at two outdoor locations in the centre of Rome. The first session took place in, Circo Massimo and the second session in the nearby neighbourhood Aventino. At both locations UC-1 was evaluated and a pair of simulation scenarios (play outs) were defined for the participants to follow.

Figure 16: First location of the simulation activities: Circo Massimo

The action was carried out in two sessions to analyse the different behaviour of sensors in open area (Circo Massimo) and among buildings (neighbourhood Aventino). The area extension and the duration of the pilot action was based on the number of available sensors. A movie describing the development of activities was prepared to disseminate the activity.

The participants, were among the citizens who have taken part in the preparatory meeting and Police Officers divided into four groups:

1. "robbery victims" - Citizens; 2. "thieves" – Police Officers; 3. "cops" – Police Officers; 4. "citizens" - Citizens.

The members of the first three groups were recognizable by means of three different colours bibs, in which the logo "City.Risks" was printed. The number of "robbery victims" equalled the number of available sensors. The number of "thieves" was about double that of "robbery victims".

The number of "cops" was about the number of "thieves". The "citizens", on foot or by bicycle, turned freely into the action area and had active the City.Risks APP with the on-board Bluetooth, acting as repeaters for sensors. The Local Police Operations Centre was able to receive the signals and display it on a map. The communication between the “cops” and the Operation Centre was through the Tetra Communication System of the Municipality Police.

The "victims of robbery" went freely in the area, recognizable for the colour of the bib that they wore. When a "victim of robbery" was stopped by a "thief", also recognizable by a specific colour bib, the "victim of robbery" delivered the sensor to the "thief". This simulated the theft. The "thief" freely moved away and, after a lapse of time (that will be defined later), the "victim of robbery" triggered the APP to signal the bicycle theft.

After a while of predefined time, the "thief" stopped. The Operations Centre signalled to the "cops" the position of the "thief". The sensor was retrieved by the "cops" and was delivered in a collection point.

At the end of the action, the recovered sensors were counted and the time need to recover them was registered.

Figure 17: Second location of the simulation activities: Aventino

The event was documented though video, and the final video was published on YouTube, see Figure 17 for link.

Figure 18: Link to the video recorded during the simulations3

3 https://www.youtube.com/watch?v=0KboAOcBPfU

https://www.youtube.com/watch?v=0KboAOcBPfU

5.3. Sofia

In Sofia, three main activities were executed; the long-term pilot activity, running throughout all of November 2017 with 15 participants, the simulations together with the consortium, and a focus group discussion, together with participants from both the short and long-term trial. The following sections outline each of these activities.

5.3.1. Simulation activities

Figure 19: Overview of icons used to represent the different actors involved in the simulations

A two-day consortium-wide meeting was scheduled in February to facilitate the execution of the simulation activities in Sofia, Bulgaria. Similar to the consortium

meeting in Waltham Forest, the first day consisted of a general project meeting and the second half focused on the testing and preparation of the simulation for the following day. The preparation consisted of going through the use-case presentation, refining the scenarios as well as testing and preparation of the infrastructure and hardware necessary for the execution of the simulation.

During the simulation, UC1, UC3 and UC4 were executed together

with the participants at a local shopping centre. Here, two different scenarios were acted out so as to evaluate the functionality of the City.Risks platform. This was

Figure 20: Roles invovled in UC-1 and UC-3

followed by a focus group discussion with the participants, in order to collect feedback on their experience, the mobile application as well as the overall concept.

Figure 21: Timeline and activities of the different roles for UC-1 and UC-3

Figure 22: Top and second floors of The Mall where the simulations were done

UC4 - reporting on ongoing events - included selected participants being identified as rioters, walking around the shopping centre. The remaining participants were designated as concerned citizens who were also walking around the mall. Whenever the concerned citizens spotted the rioters, they were instructed to use the City.Risks mobile app to report the events to the operational centre. Other participants, could then follow up with reports on how the situation was developing, for example, as the rioters were moving around the shopping centre. At the Operational Centre the situation was monitored as the status updates were received.

Figure 23: Conceptual timeline of events applicable to UC4

5.3.2. Feedback on Simulations and Long-term Pilot

An interactive usability discussion was done as part of a focus group interview during the Sofia pilot simulation. While further details are provided in D6.2, the sections below discuss the feedback provided by the focus group on usability and technical aspects of the App. This analysis focuses on general impressions and overall recommendations regarding the future of the mobile application.

A shared impression throughout the participants was that they overall really liked the concept and the overall mobile application as well as felt that the mobile application was relevant for them due to past events of bike theft. There were however some interesting comments about the usage of the application, or at least envisioned usage, and especially with regard to why a person may wish to install the mobile application in the first place. A surprising observation was that some of the participants expressed that users would install the application, mainly due to wanting to know what was happening around them, comparing it with rubbernecking.

The participants also raised some concerns regarding the overall concept, mentioning issues such as what could happen if underaged would be using the mobile application or general misuse by other users; in particular, what if other users would use the application in a similar way as some criminals use police radios to track police activity or to generate fake information to misguide other ongoing activities.

The focus group also have some interesting ideas and suggestion about how the application could be improved both in a short term and a long-term. A sough for feature was for example to have all the crime statistics available locally, on the mobile device, basically similar functionality as is provided by UC6. Another general feature request was, besides general usability improvements, that it would be more efficient to submit various events, e.g., easily made possible through voice, for situation where efficiency was important. Finally, a common recommendation was to integrate the provided functionality into other services, or vice versa, such as TripAdvisor, such that they could complement each other, as well as the acknowledgement that a significant community would be necessary for the application to be sustainable and a large enough community would be needed.

6. Pilot Activity Monitoring

Throughout the piloting activities, various statistics regarding the users were monitored across a broad range of metrics through Firebase4. Firebase enables the monitoring of both specific mobile app activities executed by the users, such as which features of the mobile application are being used and how often, as outlined in D6.1. This also provides a broad range of contextual information about the users, including demographics, their country of origin etc. These two data sets, are, however, provided independently, and are not correlated, and the latter is provided at an aggregate level, such that the information is fully anonymised and cannot be linked to an individual.

This section presents these various statistics from the pilot activity. Unless mentioned, the reporting period is between August 2017 and February 2018, beginning and end, respectively.

6.1. Demographics, affinity and geographical location

Information about individual participants was collected through the surveys disseminated during the piloting activities, thus enabling correlation across answers, as discussed in D6.2. However, not all users of the mobile application had the opportunity to respond to the surveys, since they were not necessarily recruited as part of the piloting activities and had simply downloaded the mobile application in response to the various dissemination efforts of the City.Risks project team, why the number reflected below do not necessarily reflect the ones reported in D6.2

Firebase provides general statistics and distributions about user groups and mobile app audiences regarding their interests, age group, gender and geographical location. Table 2 shows the distribution of the interests of the users.

Table 2: Top seven user categories grouped by their interests

Category % Users

Technology/Technophiles 16,6%

Technology/Mobile Enthusiasts 11,1%

Media & Entertainment/Music Lovers 10,5%

Media & Entertainment/Movie Lovers 10,1%

Technology/Social Media Enthusiasts 7,3%

Media & Entertainment/Music Lovers/Pop Music Fans 6,5%

Media & Entertainment /Gamers/Casual & Social Gamers 4,8%

Lifestyle & Hobbies/Business Professionals 4,3%

4 https://firebase.google.com/

Unsurprisingly, the primary interests represented are related technology on mobile technology in particular, reflecting that it is challenging to attract technology agnostics to try out the mobile application in the first place, and that additional effort needs to be invested in order to attract these groups.

The figure below, Figure 24, shows the geographical distribution mapped out, showing that, while the majority of the mobile app users are centred around the three pilot sites and in areas where the mobile application was disseminated as part of various events, selected other nationalities have also shown interesting in trying out the app.

Figure 24: Geographical distribution of mobile app users

Figure 25 shows the distribution of the geographical location of the mobile application users origin as a percentage of all users for the selected duration.

Figure 25: Geographical distribution of mobile app users in percentages

6.2. Gender Perspective

Throughout the City.Risks project, there has been a focus on targeting women though the various use-case provided by the platform; e.g., exemplified in a early use-case, Women’s Safety, where a user could inform the system that they were traveling from

A to B, and the system would inform potential companions such that they could help them getting there safely. Similarly, one of the participant groups that was discussed as a key participant in the long-term pilot was a women's group in Waltham Forest. The statistics illustrated in Figure 26 and Figure 27, show, however that females were underrepresented during the time in which the mobile application was made available.

Figure 26: Age and gender distribution of participants - Light blue represents female

For example, around 25% of all users were female, as illustrated in Figure 27, while the distribution in Figure 27 shows that the gender distribution is quite uneven, especially when it comes to people aged 45+.

Figure 27: Gender Distribution

6.3. User Activity

As discussed earlier, the data that is provided by the Firebase installation is rather coarse, so highly detailed information cannot easily be extracted from the system. However, Figure 28 provides an overview of the how active the users of the mobile application have been over the duration of the pilot. The figure shows the number of active users, within three given periods; Within 30 days, one week and one day. Showing that, given the purpose of the mobile application, a reasonable amount of usage was generated by the users.

Figure 28: Daily user activity over time, blue, purple and turquoise reflect month, week and day windows.

6.4. User-activity Reported by the Roma Capitale Social Media Pages.

In Rome, the pilot dissemination activities were tracked while the dissemination was ongoing, reflected in the chart show in Figure 29, showing total of 104 user impressions during the first days. Figure 29 provides the logged social media activity of the Twitter and Facebook pages of Roma Capitale, showing that the activities described in Section 4.4, reached around 100 impressions in one day alone.

Figure 29: Page visits of the social media activity in Rome - Facebook and Twitter

6.5. Analysis of Firebase data

Firebase provides generic recordings regarding specific user activities - how many times the application has been opened, how many times a view was opened while using the application - across a range of categories; age group, mobile device model and Android version. For this section, we have extracted the baseline information

provided by the tool and analysed it further. Please note that the granularity of the data provided is quite coarse, and the data does not distinguish between the users, where they are from and what role they played during the time the mobile application was available to the public. That is, these data include both people that were active participants during the pilots, interested 3rd parties that downloaded the data during conferences to try it out or did so during internal usability testing.

Figure 30: Users per age group

Figure 30 shows the numerical count of participants across four age groups. While not strictly interesting, the values are relevant as a baseline of the remaining statistics that are presented.

Figure 31: Average screen views per users

Average screen views per user: These values provide an indication of how much each individual used the mobile application during the time when the mobile application was installed on their device - or at least during the duration of the measurement period.

Average sessions per user: This metric measures the average number of times that each user opened the mobile application. In contrast to the average number of screen views, which express users’ overall activity, average sessions per user shows how many times the application was opened over the entire duration, per user.

Figure 32: Average sessions per user

Average views per session: The last statistics provided for the different age groups reflects the average activity of the users per session, that is the number of different screens the user opened while the application was open, providing an indication of how long the application was used in each session. Given that most users were either testing the mobile application, this is reflected in the data, where each user would navigate around the mobile application in order to figure out what it is doing.

Figure 33: Average views per session

6.6. Geographical Distribution of Survey Participants

Finally, the distribution of the distribution of the answers from the surveys are documented in this section. This information was extracted from the survey platform, based on Limesurvey, and was recorded when the users entered the survey link that was provided to them, either as a push notification, directly into the mobile application, or when they were forwarded to them directly by either of the pilot site partners by email.

Figure 34: Distribution of completed answers of Survey 1 per country

Please note that the data documented in the two figures, Figure 34 and Figure 35, only reflect the surveys disseminated to the UK and the Bulgarian version of the mobile application, and is not reflected in the survey efforts done in Rome, since these surveys were done offline. Further, only locations that could be identified are included, meaning that the numbers provide an indicator, and are not necessarily exact.

Figure 35: Distribution of completed answers of Survey 3 per country

6.7. Issue tracking

In order to efficiently facilitate the user-feedback that was collected across both internal testing, during the simulations and from all external parties exposed to the three main user-facing components of the City.Risks Platform, a bug tracking system was set up where all inputs were collected. These inputs were then categorized according to their importance, regarding how essential they were for the piloting activities to continue, or if they are nice-to-have, as in what would be interesting to do if the implementations should be used commercially. Essentially, working as a bug and feature tracker.

Figure 36: Tracking of usability feedback

7. Conclusion

In this report we outlined and summarised the main piloting activities that were done during the City.Risks project. Specifically, this report documented the initial plan, as it was envisioned before the pilot phase started, as well as the alternations that were performed along the course of the piloting phase as to accommodate the inputs from pilot participants, locals at the pilot sites and external stakeholders. This is done in order to provide an overview of what was planned and how the plan materialised. This was followed by an overview of the various communication channels and dissemination materials that were used across the three pilot sites as well as an elaboration of the practical activities at each pilot site.

Challenges experienced during the pilots: Some of the key challenges that were experienced during the piloting of the City.Risks mobile app was to engage a significant number of users, even when multiple attempts were made that modified the duration, the incentives and the use-cases themselves, in order to make the piloting activities simpler and more attractive. A key lesson learned here, coming both from potential participants, stakeholders, is that the communication with the potential pilot participants needs to be extremely clear, should not make assumption nor be esoteric. To a large extent, the conflict experience during the pilots resulted from the controversial topic that was to be evaluated, as the name states - City.Risks - and attracting participants to try it out. While attempts were made to mitigate these concerns, it was still challenging to achieve enough critical mass to make the mobile application self-sustainable enough to provide enough incentive for the users to participate, which was the main motivation of defining core-groups in each of the pilot sites. The motivation was not made easier by the topic of the City.Risks project.

Collaboration between Institutions and Citizens: It is evident from the outcomes of the pilots that there is a need for stronger cooperation between citizens and the institutes that are primarily responsible for the safety in cities for the successful execution of a pilot and building the necessary trust and collaboration. Government agencies can maintain this trust that is necessary for a security solution to have the credibility among citizens for the prevention, deterrence and protection from criminal events. The trust is something beyond technology and technologies like City.Risks cannot be standalone especially in the age of social networking to actually engage the citizens to use the technology and even better to report suspicious activities.

Recommendations for future work: Generally, a large portion of users did wish for more usability across the various parts of the mobile application, which is reflected by that a large amount of functionality was integrated into a single application, as to provide a single interface for all the functionality developed. The feedback indicates that it would be an advantage to distribute the functionality across different branches or into other applications with related functionality, while maintain the central infrastructure to support the application. E.g., some features, such as crime maps are more suitable for people that are going to areas that are unknown to them, while others are practical on a day-to-day basis as in the features that focus on community building, or even integrating the functionality into existing eco-systems, where the functionality is a value adding feature, rather than the primary selling point. However,

a large part of the feedback was directed towards high-level aspects, such as increased transparency, such that the users a fully aware of what happens with their data.

Further, users provided feedback as to how to improve the usability and convenience of the mobile application, such as the possibility of easily adding voice memos, a functionality that is already provide, but may be further improved upon. This is especially relevant in scenarios where speed is important, if the system is to be used in critical applications. Finally, adding additional information could provide value-adding context to what the users sees on the screen, and make the application more intuitive.

8. Abbreviations

Table 3: Abbreviations

Term Meaning

UID Unique Identifier

UC* Use-case *, referring to the originally defined City.Risks use-cases

9. Final Version of Questions and Questionnaires

9.1. Focus Group Discussion Guide - Second Version

The proposed focus groups are intended to follow on from the long-term pilot trials in the three pilot cities. The focus groups are aimed at eliciting qualitative insights into the experiences and views of participants who used the City.Risk app during the trial period. Whilst the three surveys (pre, during and post-app use) will provide ‘live’ snapshots of user experiences and a feedback mechanism throughout the trials, it is intended that the focus groups in the post-trial phase will allow users to retrospectively and interactively discuss their experiences with other City.Risks app users.

Focus groups are participant-led discussions, facilitated by a loose set of questions, themes or probes. These questions, themes or probes only provide a loose structure to the group discussion (and ensure uniformity of topic across groups/sites). The focus group participants themselves steer how and where that conversation goes.

Essentially, focus groups look at two things:

1) Content - researchers can explore what is being discussed (i.e. what participants say about their use of the app).

2) Interaction – researchers can explore how it is being discussed, in terms of group dynamics (i.e. is there consensus in the group or is one person monopolising the discussion and influencing/silencing other views).

Focus groups are a particularly useful way to discuss sensitive topics (e.g. the fear of crime) in a group context. It can produce a forum to share experiences; seek support and challenge views.

On a practical note, each focus group should be made up of no more than 8 participants and should be overseen by 1-2 facilitators. The role of the facilitator is to guide the discussion. The focus groups should be digitally recorded and transcribed. Rome and Sofia would also need to translate their transcripts into English. Each participant would have to read an Information Sheet and sign a consent form. Participation is voluntary and all transcripts will be anonymised. Given the time required (around 1 hour per focus group), the number of focus groups conducted will be dependent on how many participants and facilitators can be recruited/deployed.

Key themes/questions that the focus groups will cover:

- User experiences using the app (i.e. user friendliness; access to help/support from the City.Risks team when needed)

- Design features (i.e. how it appeared; aesthetics) - Favourable/ less favourable aspects of the app - Aspects of the app that lessened/ added to user fears of crime/perceptions of

safety - Any unintended consequences of app use - Any particular fears about crime/ safety that are not addressed by the app

- How would users improve the existing app (i.e. features they would add/ remove/ change?)

- Would users recommend the app to friends/family

9.2. Final versions of the surveys

The following sections provide the English version of three surveys that were disseminated to the participants as part of the long-term pilot. The formatted used included various nomenclatures, which are defined in Table 4: Symbols and their meaning.

Table 4: Symbols and their meaning

Term Meaning

* Implies an mandatory question

[ ] Implies that the user could provide a written statement as an answer

Implies a checkbox where only one option may be selected – Radio button

Implies a checkbox where multiple options may be selected – Checkbox

9.2.1. City.Risks Survey 1

This survey is part of a Horizon 2020 project, entitled City.Risks: Avoiding and

Mitigating safety risks in urban environments.

All your answers are confidential and the responses you provide will be stored on a

secure server.

Please read the questions carefully before answering. The survey will take 5-10

minutes to complete.

By participating in the survey you agree that:

● I understand that if I decide at any time during the pilot trial that I no longer

wish to participate in this project, I can notify the organisers of the pilot and

withdraw from it immediately without giving a reason. I understand that I will

also be able to withdraw my data at any point up to January 2018.

● I consent to the processing of my personal information for the purposes

explained to me. I understand that such information will be treated in

accordance with the terms of the Data Protection Act 1998.

Thank you for taking the time to participate in this survey.

There are 19 questions in this survey

User ID

The information provided here is used to link the application activity with answers

provided in the online survey throughout the pilot.

[]If not already filled, please provide your City.Risks ID *

Please write your answer here:

You may find your City.Risks ID in the City.Risks mobile app in the profile menu (top

left corner), just below your name.

Demographic information

[]

How old are you?

*

Only numbers may be entered in this field.


●

[]

How would you define your gender?

*

Choose one of the following answers

Please choose only one of the following:

● Female

● Male

● Transgender

● Other

● Prefer not to answer

[]

Do you live with children under the age of 16?

*



● Yes

● No


[]

How long have you been living in Waltham Forest?

*



● Less than 1 year

● 1-2 years

● 3-4 years

● 5-10 years

● 10 or more years

● I don’t live in this neighbourhood

[]

How long have you been working in Waltham Forest?

*



● Less than 1 year

● 1-2 years

● 3-4 years

● 5-10 years

● 10 or more years

● I don’t work in this neighbourhood

Feelings of safety, fear of crime, and victimisation

experiences

The following section will ask you about your perception of safety, fear of crime and

any experiences of victimisation.

[]

In general, how worried are you about being a victim of a crime?

*



● Very worried

● Fairly worried

● Not very worried

● Not at all worried

● Don’t know

[]

How much is the quality of your life affected by fear of crime?

*



● Not at all

● Slightly

● Moderately

● Very

● Extremely

● Don’t know

[]

Do you avoid any areas, streets or places in your neighbourhood due to

fear of crime?

*



● Never

● Sometimes

● Often

● Very often

● Don’t know

[]

How safe do you feel walking around alone in your local area during the

day?

*



● Very safe

● Quite safe

● Neither safe, nor unsafe

● Quite unsafe

● Very unsafe

● I never walk around alone during the day in my area

● Don’t know

[]

How safe do you feel walking around alone in your local area after dark?

*



● Very safe

● Quite safe

● Neither safe, nor unsafe

● Quite unsafe

● Very unsafe

● I never walk around alone at night in my area

● Don’t know

[]

Do you ever change your behaviour because you are concerned about

being a victim of a crime?

*



● Never

● Sometimes

● Often

● Very often

● Don’t know

[]

Thinking about your local area, are you worried about being a victim of

any of the following crimes?

*

Check all that apply

Please choose all that apply:

● Harassment or bullying

● Antisocial Behaviour

● Gang related criminality

● Motor vehicle theft

● Bicycle theft

● Pickpocketing

● Burglary

● Robbery or mugging

● Vandalism

● Traffic incidents

● Assault

● Sexual assault

● Cyber crime

● Identity theft and/or fraud

● Don’t know


● None

● Other:

[]

Thinking about your local area, are you worried that anyone close to you

(for example a family member, a partner, or friend) will be a victim of any

of the following crimes?

*







● Bicycle theft

● Pickpocketing

● Burglary


● Vandalism


● Assault

● Sexual assault

● Cyber crime


● Don’t know


● None

● Other:

[]

Thinking about your local area, which of the following crime / problem

are not dealt with successfully by responsible authorities?

*







● Bicycle theft

● Pickpocketing

● Burglary


● Vandalism


● Assault

● Sexual assault

● Cyber crime


● Don’t know


● None

● Other:

[]

Have you been a victim of any of the following crimes in the last 12

months?

*



● I have not been a victim of a crime in the last 12 months





● Bicycle theft

● Pickpocketing

● Burglary


● Vandalism


● Assault

● Sexual assault

● Cyber crime


● Don’t know


● Other:

Usage of smartphones

[]

How often do you use social media (e.g. Facebook, Twitter, etc.)?

*



● Never

● Less than once a month

● 1-3 times per month

● Every week

● Almost every day

● Once every day

● Several times per day

[]

How often do you use any mobile applications or web based tools

specifically because you would like information related to safety (e.g.

safety apps, crime statistics etc)?

*



● Never

● Less than once a month

● 1-3 times per month

● Every week

● Almost every day

● Once every day

● Several times per day

[]

Which of the existing services of the City.Risks app do you think could be

helpful for increasing your feelings of safety?

*



● None

● Alternative routing

● Crime maps of your city

● Sharing and obtaining information about ongoing incidents

● Sending and responding to requests for witnesses of crimes

● Don’t know

● Other:

Thank you for completing our survey.

Submit your survey.

Thank you for completing this survey.


This survey is part of a Horizon 2020 project, entitled City.Risks: Avoiding and Mitigating safety risks in urban environments.

All your answers are confidential and the responses you provide will be stored on a secure server.

Please read the questions carefully before answering. The survey will take 5-10 minutes to complete.



Meta

[]


During pilot

[]Have you used the City.Risks app? *


Yes

No

[]

How would you rate your overall experience of using the City.Risks app so far? Please provide reasons for your answer in the box below.

Only answer this question if the following conditions are met:

Answer was 'Yes' at question '2 [DurHaveYouUsedApp]' (Have you used the City.Risks app?)



Very poor

Poor

Neither poor, nor good

Good

Very good

Make a comment on your choice here:

[]Please provide reasons for your answer


[]

Which services of the City.Risks app did you find useful?


((DurHaveYouUsedApp.NAOK == "Y"))



None

Alternative routing

Crime maps of your city

Sharing and obtaining information about ongoing incidents

Sending and responding to requests for witnesses of crimes

[]

Which services of the City.Risks app have you liked?



https://cityrisks.limequery.com/admin/questions/sa/view/surveyid/632628/gid/5/qid/180




None

Alternative routing




[]Please provide reasons for your answer.


[]

Are there any services of the City.Risks app that did not work properly (e.g. technical difficulties, lagging etc.)?





I haven’t used any services

All services of the of the City.Risks app worked properly

None of the services of the City.Risks app worked properly

Alternative routing




Web portal

Registration

Other

[]Please describe the nature of the technical difficulties you experienced



[]

Whilst using the City.Risks app have you encountered any of the following?





Haven’t used the app

Inappropriate posts of images by other users

Inappropriate language use / comments by other users

Fake reports of incidents

Other:

[]

Has the City.Risks app increased your perception of safety?





Not at all

Not very much

To some extent

To a large extent

Don’t know

[]

In what ways can the City.Risks app be improved to encourage greater use?


[]

Have you visited the City.Risks web page for your city?




Yes

No


Submit your survey.



This survey is part of a Horizon 2020 project, entitled City.Risks: Avoiding and Mitigating safety risks in urban environments.

All your answers are confidential and the responses you provide will be stored on a secure server.

Please read the questions carefully before answering. The survey will take 5-10 minutes to complete.



After pilot

[]Have you used the City.Risks app? *


Yes

No

[]

How would you rate your overall experience of using the City.Risks app?


Answer was 'Yes' at question '1 [ApHaveYouUsedApp]' (Have you used the City.Risks app?)



Very poor

Poor

Neither poor, nor good

Good

Very good

I never used any functionalities. Please state why you haven’t used any functionalities in the box below.


[]Which services of the City.Risks app did you find useful?



None

Alternative routing




[]Which services of the City.Risks app have you liked?



None

Alternative routing



Sending and responding to requests for witnesses of crime

[]Please describe why you have liked or disliked any service(s).


[]Are there any services of the City.Risks app that did not work properly (e.g. technical difficulties, lagging etc.)?



I haven’t used the app

All of the services of the City.Risks app worked properly

None of the services of the City.Risks app worked properly

Alternative Routing




Web portal

Registration

Other:

[]Please describe the nature of the nature of the technical difficulties you experienced


[]

Whilst using the City.Risks app have you encountered any of the following inconveniences?



Inappropriate posts of images by other users

Inappropriate language use / comments by other users

Fake reports of incidents

I haven’t used the app

None

Other:

[]Has the City.Risks app increased your perception of safety?



Not at all

Not very much

To some extent

To a large extent

Don't know

[]Did you like using the alternative routing app service? Please provide reasons for your answer in the box below.


((ApHaveYouUsedApp.NAOK == "Y"))



Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]To what extent has the alternative routing app service increased your feelings of safety? Please provide reasons for your answer in the box below.







Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]Did you like using the crime maps of your city app service? Please provide reasons for your answer in the box below.





Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]To what extent has the crime maps of your city function increased your feelings of safety? Please provide reasons for your answer in the box below.







Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]

Did you like using the app function in which you could share and obtain information about ongoing events? Please provide reasons for your answer in the box below.





Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]


To what extent has the app function in which you could share and obtain information about ongoing events increased your feelings of safety? Please provide reasons for your answer in the box below.





Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]

Did you like using the app function in which you could send and respond to requests for witnesses of crimes? Please provide reasons for your answer in the box below.





Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it




[]

To what extent has the app function in which you could send and respond to requests for witnesses of crimes increased your feelings of safety? Please provide reasons for your answer in the box below.





Not at all

Not very much

To some extent

To a large extent

Don’t know

I haven’t used it


[]

In general, has the City.Risks app increased your feelings of safety in Waltham Forest? Please provide reasons for your answer in the box below.



No, not at all

No, not very much

Yes, to some extent

Yes, to a large extent


Don’t know


[]

In general, has the City.Risks app made you more concerned about crime in your city? Please provide reasons for your answer in the box below.



Not at all

Not very much

To some extent

To a large extent

Don’t know


[]

Has the City.Risks app made you more concerned about crime in any specific neighbourhood in your city? Please provide reasons for your answer in the box below.



Not at all

Not very much

To some extent

To a large extent

Don’t know


[]

Do you have any suggestions on how to further develop the City.Risks functionalities so that they are of greater use to you?


[]

Would you use the City.Risks app after your participation in this pilot trial has ended?



Yes

No

Maybe

Don’t know

Prize and Personal Details

[]Do you wish to participate in the lottery and/or the focus group discussion?



Do you wish to participate in the prize lottey?

Do you wish to be invited for the focus group discussion? If you wish to participate in either (or both), you will be asked for your personal details on the next page.

[]Please enter your personal details, such that we may contact you.


((METAParticipation_FocuGroupParticipati.NAOK == "Y") or (METAParticipation_PrizeParticipation.NAOK == "Y"))



Name

Email

Phone Number


Submit your survey.


Overall Evaluation of the City.Risks...

Documents

Transcript of Overall Evaluation of the City.Risks...