Report on first prototype of Building Block Repository...D202.2 Report on first prototype of...

63
Ecosystem infrastructure for smart and personalised inclusion and PROSPERITY for ALL stakeholders Report on first prototype of Building Block Repository (2 nd iteration) Project Acronym Prosperity4All Grant Agreement number FP7-610510 Deliverable number D202.2-2 Work package number WP202 Work package title Building Block Set Authors Till Riedel, Lukas Smirek, Chris Veigl, Martin Deinhofer, Konstantinos Votis, Nikolaos Kaklanis, Aggeliki Konstadinidou, Dimitrios Tzovaras, Charalampos Karagianidis, Anastasia Cheetham Status Final Dissemination Level Public Delivery Date 15/03/2017 Number of Pages 62

Transcript of Report on first prototype of Building Block Repository...D202.2 Report on first prototype of...

Page 1: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

Ecosystem infrastructure for smart and personalised inclusion and PROSPERITY for ALL stakeholders

Report on first prototype of Building Block Repository

(2nd iteration)

Project Acronym Prosperity4All Grant Agreement number FP7-610510

Deliverable number D202.2-2

Work package number WP202 Work package title Building Block Set

Authors Till Riedel, Lukas Smirek, Chris Veigl, Martin Deinhofer, Konstantinos Votis, Nikolaos Kaklanis, Aggeliki Konstadinidou, Dimitrios Tzovaras, Charalampos Karagianidis, Anastasia Cheetham

Status Final Dissemination Level Public

Delivery Date 15/03/2017 Number of Pages 62

Page 2: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

2

Abstract

Three years have passed in which components have been isolated, documented and gathered within the Building Block Repository that is part of the GPII Developer Space. This software deliverable prepares the first public release of this software that will be available as part of a web site under http://developerspace.gpii.net. While the software itself is the main deliverable this reports tries to outline the objectives of building this software. Much of this work was guided by the overall architecture and multiple user evaluations. On this basis we try to explain certain design decisions we took over the course of implementing an essential part of the GPII infrastructure.

The current document further contains a complete list of building blocks (software and hardware component as well guidelines and tutorials) that have been provided to bootstrap the effort as part of the EU funded Prosperity4ALL project. Instead of replicating online information we mostly link the source repository that contains further information and current progress about single components.

This is the second iteration of a rolling report. The headlines of the sections that contain major changes in content over the last iteration are highlighted with yellow background color and marked with an asterisk gyph (*).

Keyword List

Repository, Listing, Building Blocks, Developers, Components, Developer Space, Accessibility

Page 3: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

3

Version History

Revision Date Author Organisation Description

1 07/12/2016 Till Riedel KIT 1st version including TOC.

2 07/01/2016 Till Riedel, Lukas Smirek, Chris Veigl, Martin Deinhofer, Konstantinos Votis, Nikolaos Kaklanis, Aggeliki Konstadinidou, Dimitrios Tzovaras, Charalampos Karagianidis, Anastasia Cheetham

CERTH,FHTW,HDM,IDRC,KIT

Input via the DSpace Prototype

3 14/01/2016 Till Riedel, Lukas Smirek, Chris Veigl, Martin Deinhofer, Konstantinos Votis, Nikolaos Kaklanis, Aggeliki Konstadinidou, Dimitrios Tzovaras, Charalampos Karagianidis, Anastasia Cheetham

CERTH,FHTW,HDM,IDRC,KIT

Additional Input on Work done as part of P4A

4 14/02/2016 Till Riedel KIT Integrated DSpace Screenshots and eval feedback

5 26/02/2016 Lukas Smirek HDM Additional Input URC Docs &Guidelines

6 26/02/2016 Till Riedel KIT Updated Content from online description, including categorization and Images. Formatted correctly

7 29/02/2016 Till Riedel KIT Added text describing interaction with SP1 and SP4. Added conclusion

8 04/03/2016 Stefan Schürz, Alfred Doppler

LFTL Internal Review, Typo fixes

9 15/03/2016 Andreas Stiegler HDM Internal Review

10 15/03/2016 Till Riedel KIT Edits according to review

Page 4: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

4

Revision Date Author Organisation Description

11 16/02/2017 Till Riedel KIT Second iteration Made document robust to changes in production URLs and freeze Moved progress reports on single components out of the document where they can be kept up to date Addition of sections 1.4, 1.5, 2.2.3,2.6 Change of executive summary and conclusion Resorting of section 3 to match section 2 Contains complete list of components from WP202 of P4A Updated repository links

Page 5: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

5

Table of Contents

Executive Summary .......................................................................................................... 8

Glossary............................................................................................................................ 9

1 The Role of the Building Block Repository ..............................................................11

1.1 Purpose and structure of this document .................................................................. 12

1.2 Purpose of Building Blocks in the GPII Developer Space ........................................... 14

1.3 Repository Creation and Management Strategies .................................................... 17

1.3.1 Service Design, Use Cases and Personas ............................................................ 18

1.3.2 Categorization and Structuring in a first Prototype ........................................... 21

1.3.3 Considerations for the second prototype of the repository * ........................... 26

1.3.3.1 Reduction categorization schemes that conflict with quality * ......................... 27

1.3.3.2 Cognitive Funneling with good READMEs * ....................................................... 27

1.3.4 Planned final refinements the release * ............................................................ 28

1.3.4.1 Remove categorizations of Code * ..................................................................... 28

1.3.4.2 Lowering the cost for contribution * ................................................................. 29

1.3.4.3 Added value through expert categorization * ................................................... 29

1.3.4.4 Sort by quality metrics * .................................................................................... 29

1.3.4.5 Incentives to contribute * .................................................................................. 30

1.4 Architectural Integration of the Building Blocks Repository * .................................. 30

1.4.1 Integration with the P4A Quality Infrastructure and external Repositories and curated component listings * ............................................................................................ 30

1.4.2 Integration with the P4A Feedback Mechanisms and Gamification * ............... 32

1.4.3 Integration with the P4A Security Component and other components of P4A * 34

1.4.4 Integration with the Development Environments * .......................................... 34

2 List of contributed Building Blocks by Task .............................................................36

2.1 External Building Blocks and online collection (FHTW) ............................................. 36

2.2 AT Specific I/O Modules (P4A T202.2) ....................................................................... 40

2.2.1 AsTeRICS AT Modules (FHTW) ........................................................................... 40

AsTeRICS Packaging Environment (APE) ...................................................................... 41

Page 6: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

6

Managing Licenses of the AsTeRICS Building Blocks .................................................... 41

2.2.2 Websocket Interface for AsTeRICS Building Blocks (FHTW) .............................. 42

2.2.3 RoboBraille.Web.API (Sensus) * ......................................................................... 42

2.3 Generic Multimodal Interaction Modules (P4A T202.3) ........................................... 43

2.3.1 Arduino UNO: Microcontroller Interface (I/O Transducer Modules) (FHTW) ... 43

2.3.2 Universal HID Actuator: Mouse/Keyboard/Joystick Emulation (FHTW) ............ 44

2.3.3 Camera Input Module: FacetrackerLK, XFacetrackerLK: (FHTW)....................... 44

2.3.4 P4A Haptic Module (CERTH) ............................................................................... 45

2.3.5 P4A Android Vibration Module (CERTH) ............................................................ 46

2.4 Smart Device and Environment Modules (P4A T202.4) ............................................ 46

2.4.1 EnOcean: Smart Home Integration (FHTW) ....................................................... 46

2.4.2 Point and Control: Spatial Interaction for Remote Control (KIT) ....................... 48

2.4.3 IndianaJS: Spatially aware Web of Things (KIT).................................................. 48

2.4.4 Context-WiFi Login (KIT) ..................................................................................... 49

2.4.4.1 Universal Remote Console Socket Templates .................................................... 49

Wöhlke Web Electricity Socket Template 1.0 .............................................................. 50

Plant Sensor Templates 1.0 .......................................................................................... 50

NetAtmo Templates 1.0 ............................................................................................... 50

Coloured light bulb Templates 1.0 ............................................................................... 51

DVD Player Templates 1.0 ............................................................................................ 51

TV set 1.0 ...................................................................................................................... 51

2.5 Real-Time User Monitoring Modules (P4A T202.5) .................................................. 52

2.5.1 P4A SensorAPI (KIT) ............................................................................................ 52

2.5.2 FallDetectionModule (CERTH) ............................................................................ 53

2.5.3 P4ALL Affect Sensing Module (CERTH)............................................................... 53

2.5.4 Social Network Interaction Module (CERTH) ..................................................... 54

2.5.5 jActivity (KIT) ...................................................................................................... 55

2.5.6 Open BCI: Bioelectric Signal Acquisition and Processing (FHTW) ...................... 55

2.6 Adaptive Web (P4A Task 202.6) * ............................................................................. 56

Page 7: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

7

2.6.1 Adaptive Web Components (HDM/UStutt) *..................................................... 56

2.6.2 CSS Context Query (UStutt) * ............................................................................. 56

3 Tutorials, Guidelines and Handbooks .....................................................................57

3.1 Tutorials ..................................................................................................................... 57

3.1.1 URC Tutorials ...................................................................................................... 57

3.1.1.1 Tutorial: Getting started with URC (HDM) ........................................................ 57

3.1.1.2 Tutorial: Creating Sockets with the SocketBuilder (HDM) ................................. 57

3.1.1 Asterics Tutorials * ............................................................................................. 58

3.1.1.1 AsTeRICS Plugin Development - Step by Step (FHTW) * .................................... 58

3.1.1.2 AsTeRICS Camera Mouse Model Creation - Step by Step (FHTW) * .................. 58

3.1.1 Fluid/FLOE Tutorials ........................................................................................... 58

3.1.1.1 User Interface Options Tutorial .......................................................................... 58

3.1.1.2 Infusion Preferences Framework Tutorial ......................................................... 59

3.1.1.3 First Discovery Tool Tutorial ............................................................................... 59

3.2 Guidelines & Handbooks .......................................................................................... 60

3.2.1 URC Guidelines ................................................................................................... 60

3.2.1.1 Design Guidelines for User Interface Sockets .................................................... 60

3.2.1.2 Making a business case with URC ...................................................................... 60

3.2.1.3 URC Frequently Asked Questions ....................................................................... 60

3.2.1 Learning Material ............................................................................................... 61

3.2.1.1 Accessible Standardized Testing Guide .............................................................. 61

3.2.1.2 Inclusive EPUB 3 Guide ....................................................................................... 61

3.2.1.3 Accessible Web Games and Interactive Simulations ......................................... 62

4 Conclusion and Future Work * ...............................................................................63

Page 8: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

8

Executive Summary

This document presents the recently refined version of the Building Blocks Repository which is an essential part of the GPII Developer Space. The update of this document adds a few parts that were planned but missing in the last iteration. More importantly it, however, reports on the important transition towards sustainable operation of the repository within the GPII.

As this document is an accompanying report to a software deliverable, we encourage the reader to follow the links to the online version for up to date information (http://staging.developerspace.gpii.net). The GPII Developer Space prototype includes many more Building Blocks than those that are listed here. We primarily included items in this document that were part of active development within the first 3 years of the project span. Many of the Building Blocks have a history and scope beyond the project but were refined and made available with the scope of specific objectives of Prosperity4All. The three parts of the document describe the creation of a prototype repository in the scope of the overall effort to create the GPII Developer Space, describes the components as well as documentation contained currently in the prototype.

The online prototype that is currently reachable via http:// staging.developerspace.gpii.net provides an alternative way to explore the contents of this deliverable. To summarize: this deliverable documents a snapshot of the ongoing work that was done as part of the Prosperity4All project for the purpose of reviewing the past 3 years of project work within work package WP202. This documents milestone for the project that finalizes an important phase, however, work will continue within and beyond the project in an iterative, agile and dynamic fashion.

In this version of the document we omitted updates on single components during the reporting period. As all components are hosted on Github since the last release and in most cases refinements were done within the scope of this project, we like to direct the interested reader to follow the project progress via the provided links (either following the link “commits” for individual changes or “releases” for major releases and changelogs).

Page 9: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

9

Glossary

Head Head

Global Public Inclusive Infrastructure (GPII)

Collaboratively created infrastructure that ensures that everyone who faces accessibility barriers can access and use information technologies.

Prosperity4All (P4A) Project funded under Contract Number 610510 within the European Union's 7th Research and Innovation funding programme (FP7). P4A builds important parts of the GPII.

GPII Developer Space An ecosystem enabling set of services within the GPII that provide ease access to a comprehensive set of resources, tools, and open community infrastructure to help implementer find and use components and frameworks that have been built with personalized accessibility in mind.

Implementer Stakeholders of the GPII Developer Space providing applications, tools and services to end users adapted to their needs (term used to distinguish from developers who provide e.g. building blocks)

Building Block A term to describe reusable hardware and software components addressed towards implementers

Stakeholder A person, group or organization that are directly or indirectly involved or affected by a service.

Actor Any person involved in the creation, delivery, support, or use of a service. In case of the GPII Developer Space those are mostly developers of components, authors of other content and implementers.

Need A necessary and/or desired function or condition.

Entry Point Instances of access to a service, where actors are able to join the service as customers, providers or stakeholders. The GPII Developer Space front page, but also the listing of a single Building Block can be such entry points.

Journey Map A visual representation of a particular actors’ experience with a service.

Page 10: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

10

Head Head

Persona A representation of a user group with shared needs and characteristics. Personas are the distillation of primary research with people. A persona would be given a name and characteristics as well as needs.

Assistive Technology (AT) Any product, whether off the shelf, modified, or customized, that is used to increase, maintain, or improve the functional capabilities of someone with disabilities (also temporal).

Accessible Accessible to people with a diverse range of hearing, movement, sight, and cognitive ability.

Information technology (IT) Use of any computer aided devices, infrastructure and processes to create, process, store, secure and exchange all forms of electronic data.

Human Interface Device(HID) A computer device that interacts directly with humans (e.g. a joystick, keyboard or mouse)

Application Programming Interface (API)

Set of routines, protocols, and tools for building software provided by a Building Block.

Infrastructure Something that this is constructed to support a greater entity or system. See also Global Public Inclusive Infrastructure as an example

Platform A platform is a group of technologies that are used as a base upon which other applications, processes or technologies are developed

Ecosystem A complex system of interacting entities that depend on each other and, in balance, support each other’s existence. Ecosystems involve competition and predation, but in balance the whole ecosystem works or collapses.

Resource Information, materials, staff, and other assets that can be drawn on by a person or organization in order to function effectively.

Page 11: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

11

1 The Role of the Building Block Repository

The Building Block Repository (that you will also find as “Parts&Tools” pictured in Figure 1 within the related Web Version) is part of a global effort to build a Developer Space within the Global Public Inclusion Infrastructure (GPII). The items described in this report can thus be seen in a much broader scope as explained below (see 2.3). The Building Block Repository is not actually repository, but rather a community curated listing of code artefacts that are developed and maintained in a strictly distributed fashion. It contains any contributions of components made by the Prosperty4All project to the GPII Developer Space community, but will include also any external component that can support the effort.

Figure 1: Snapshot of the first version’s (December 2016) front page build within Prosperity4All

The purpose of contributing Building Blocks as part of the funded effort of the project is to bootstrap an ecosystem that provides developers with “Libraries, Software & Hardware Modules as well Code Snippets that will help you build accessible and personalized technology”.

It is not only the single component that should “make it easier, faster and less expensive to conceive [and] create” any inclusive implementation, but the collection as whole. It is clear that any Building Block Repository can hardly ever be complete. This deliverable, however, adds and documents a considerable amount of new possibilities to the space of assistive technology (AT) and accessible information technology (IT). Much of the work was started before Prosperity4All but was made available for the first time under permissive open source licenses and in a reusable manner as part of this deliverable. Furthermore, existing

Page 12: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

12

components developed outside the project were systematically categorized in a community effort.

With our contributions (both those we built and those which we found), we want to build a bridge to other advanced research topics in the areas of AT Interfaces, Multimodal Interaction, Cognitive Systems and Smart Spaces, Personalized Health and Active Aging, Sensing Enterprises and Connected Social Media.

With the integration of the Building Block Repository into the “GPII Developer Space”, a developer targeted service infrastructure that is part of the overall Prosperity4All ecosystem, we provide concrete entry points (the search as well as the individual Building Blocks) for those who seek to integrate innovations and bring them to use in the context of smart and personalized inclusion. Not claiming completeness by any means, we want to provide application programming interfaces and aim to stimulate the development of new intelligent modules by researchers, students, and others to rethink novel interaction and context-sensing techniques from the perspective of users with various kinds of permanent or temporary disabilities.

1.1 Purpose and structure of this document As this document is an accompanying report to a software deliverable, we encourage the reader to follow the links to the online version for up to date information. The software presented here was developed or improved as part of the BUILDING BLOCKS work package of the Prosperity4ALL FP7 EU Project. The GPII Developer Space prototype includes many more Building Blocks than those that are listed here. We primarily included items in this document that were part of active development within the first 2 years of the project span. Many of the Building Blocks have a history and scope beyond the project but were refined and made available with the scope of specific objectives of Prosperity4All (according the description of work):

• To collect and create an extensive repository of alternative input, output and processing modules, components and adapters to facilitate development of assistive technologies and accessible interfaces.

• To integrate existing assistive solutions from previous projects such as AsTeRICS and extract useful sensors, processors and actuators from it to a more general API. To integrate Human Interface Device (HID) emulators into Prosperity4All and unify them under a unified API.

• To collect, extend, integrate, and make production-ready a collection of web user interface components and web framework infrastructure that will reduce the cost and complexity of building accessible web applications and web-based assistive technology

• To foster a community of contributors, maintainers, and sustainers of these components, ensuring they will be viable over the long term.

Page 13: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

13

This document describes the common open repository of state of the art AT interface modules globally and the concrete work of the development teams within it that provide AT interface module developers the capabilities to excel beyond state of the art by providing access to novel technologies.

This report includes all developments within WP202 with some exceptions. • Work that is dependent on new Frameworks and Tools that were not available after

the first year of the project as (e.g as part of WP203 or WP204) will follow in a further iteration.

• Work on common APIs that is running in parallel to other development activities is ongoing and is only partially included in the current state of the Building Blocks. This is mostly also due to interactions with Frameworks (e.g. the integration with the Asterics REST API with the SensorAPI or refinement of WebComponent and Messaging APIs that are compatible with the architectural Integration domains, P4A Nexus / MyUI design patterns)

It will be the purpose of our second milestone and the accompanying document to particularly fulfill the remaining objectives:

• To ease integration of novel heterogeneous sensor sources into AT based on state of the art algorithms

• To enable smart and proactive adaptation of AT by sensing its usage context in terms of the users current state and the environment

• To show-case possibilities for adaption techniques based on real world interactions of IT-based AT

This Section describes design considerations and the relation to this work towards other parts of the project Section 3 is structured according to the description of Prosperity4All.

Separate boxes highlight work done as part of the project is highlighted. Further content was taken from the current snapshot of the online documentation of the delivered software prototype as of January 2016. Section 4 describes further non-software resources delivered such as handbooks and templates and tutorials in more detail. The online prototype that is currently reachable via dspace.teco.edu provides an alternative way to explore the content this deliverable. To summarize: this deliverable documents a snapshot of the ongoing work that was done as part of the Prosperity4All project for the purpose of reviewing the past 2 years of project work within work package WP202. This documents milestone for the project that finalizes an important phase, however, work will continue within and beyond the project in an iterative, agile and dynamic fashion.

Page 14: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

14

1.2 Purpose of Building Blocks in the GPII Developer Space Although is not the objective of this document to reflect on the overall GPII Developer Space architecture (this is outlined as part of deliverable D201.3) and its role in the ecosystem (this is outlined as part of D201.1), it is important to understand this work within the scope of the overall effort to create a GPII Developer Space within Prosperity4All.

Figure 2: How and for what implementers (source: D402.2)

From the perspective of Building Block developers, the developer space is a means of interacting with potential implementers.

Up to this date only few and rather isolated space exist for implementers to explore components to build AT and accessible IT. This is reflected in the result of the first survey’s with implementers (see Figure 2), that personal contact with colleagues (or project partners) or general search are the most common strategies for exploring the solution space. And our preliminary studies suggest that they mostly are looking for concrete code rather than detailed documentation. Also background interviews conducted by the SP4 team show there is scales only in very limited way today, and implementers feel great need for a community space that particularly focusses on giving them access to new building blocks (currently more and more external implementers are interviewed to support this).

At the same time developers complain about slow adoption and few possibilities to connect to potential users with a reasonable effort. This is particularly important as there is often no direct incentive to commercially market: suitable Building Blocks are often living in a research and/or open source domain. Initial input from the evaluation team suggests that motivation is seldom monetary but often the interaction with other developers.

Page 15: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

15

Figure 3: Motivation to contribute to a developer space (source: D402.2)

Figure 4: Building Blocks within the context of the project (seen on the upper left)

From the graphical project representation in Figure 4 (that also models parts of an ecosystem) it becomes clear that in contrast to implementations by the solution developers, the building blocks are addressed towards AT implementers as important part of the ecosystem. As the project tries to bootstrap the ecosystem, it is important that building blocks would reach a milestone in terms of quantity and quality early in the project. The

Page 16: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

16

collection presented here will be the basis of interaction with implementers and we hope to make relevant links through the developer space. One important milestone was reached as the current developer space web prototype that includes all the building blocks reported on here was tested with developers and potential implementers. Starting with this first internal beta release the Building Block Repository will become the primary way to share artifacts from WP202 with the rest of the P4A ecosystem.

They also interact particularly with Development Tools (WP203) and need to integrate with an overall Infrastructure Architecture (WP201). This part will be the focus of the last part of the work that will be reported in the next iteration.

Page 17: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

17

1.3 Repository Creation and Management Strategies One of the main challenges of the work on building blocks was the bootstrapping of a bottom up movement for sustainable contribution. In order to emulate “the real world” as best as we could, it was important follow the non-prescriptive path of inclusive design that is outlined as part of the SP1 work. With many developers contributing very different pieces from real projects and implementers sometimes waiting for someone to tell them what to use, it was often difficult not to choose the easy (yet not sustainable) path of top down matchmaking within the project. Instead, we started early to document the diversity of possibilities that were offered to implementers, e.g. through the wiki (see Figure 5).

In order to analyze good categorization structures we looked at number of “marketplaces” and repositories, including git hub, the google play store and stackoverflow, but also hardware listings like octopart (evaluation in SP4 also compared user preferences towards different listing styles). We also looked at particularly the added value of an accessibility focused repository. Another concern was that we did not want to replace other repositories and that “our” Building Blocks should “live" outside of our prototype as a precondition to sustainable open source development.

Figure 5: Start of the repository in the wiki until mid-2015

It became soon obvious that if the project should succeed on the premises of voluntary adoption, we would have to do much more to support the searching process of implementers and incentivize them to try out the offering that were laid out to them.

Page 18: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

18

Figure 6: Testimonial of early success with repository building... (Source: http://lists.gpii.net/pipermail/dspace/)

1.3.1 Service Design, Use Cases and Personas

When we started with the task, it was one of our first activities to try to create a better understanding of our stakeholders and the interactions we wanted to support with our service. We started off with small focus groups and persona building activities inside the work package and building first functional mockups of a repository within the first year. At the same time the first background interviews were conducted by the evaluation (SP4) team with implementers (SP3) and the developers (SP2) listing their planned contribution and their own expectations. The results werewere shared via the public project wiki (wiki.gpii.net) and an open discussion started. While working on a better fit between components and user needs, we experimented with different documentation strategies via the project wiki and soon found first success reaching our implementers (see Figure 6). Driven by the early success we created our first prototypes of a Building Block Repository (see Figure 7). However, it soon became clear that sustainability and scalability of our approach became an issue and would be important to understand the role of our Building Block repository as an entry point to a whole Prosperity4All ecosystem. After the first year of the project, we reached a point at which we understood the diversity of the contributions as well as the expectation of implementers.

Page 19: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

19

Figure 7: First functional prototype at the beginning of 2015

With the work of the ecosystem (SP1) team progressing, it became clear that the Building Block repository needed to be part of orchestrated project effort to bootstrap the GPII Developer Space (DSpace) and could not be designed without an understanding of the whole service ecosystem. At the same time it was the only possibility to test the interactions within the ecosystem already within the project. The journey map of a persona using the DSpace, that helped building the current prototype, is depicted in Figure 8. Looking at it, it becomes clear that a fully functional Building Block repository is essential for this persona.

Page 20: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

20

Figure 8: Journey map describing possible interactions with the repository (source:SP1)

The general usefulness of any repository is naturally strongly dependent on the contributed content. With more content available the usability became an increasingly important factor. Within 2015 the process of iteratively increasing both usefulness (through more content) and usability (by better navigation of this content) became one of the core schemes of our collaborative work that were channeled into a unified to design of the GPII Developer Space. Much of our experience with collecting components as part of the Wiki and communicating with implementers inside the project are present in this design.

Figure 9: Design Sketch of the Developer Space

Page 21: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

21

1.3.2 Categorization and Structuring in a first Prototype

One of the main challenges when moving from service and interaction design ideas towards a tangible prototype was an understanding about the search strategies of the users. The evaluation team (SP4) helped considerably by formatively testing our ideas and prototypes at an early stage. While personas were very helpful in the design phase to consider particular edge cases as well as to generate hypotheses about “typical” users, we external feedback on concrete designs was needed. With the milestone of creating a first internally usable version the Building Block repository as described in this document, we are for the first time able to watch implementers interacting with the offering we are making and ask them for concrete feedback. Figure 10 shows that the importance of search seems to be at least reflected in the use of the GPII Developer Space prototype by our project-internal implementers (we tried to equally support both search and exploration at any stage)

Figure 10: Exploration strategies of users in the latest Developer Space user study.

While building the prototype we had to rely on assumptions and early research on the concrete prototype. Supporting effective search and exploration, the organization and categorization became an important issue.

One cornerstone was existing efforts and experiences by partners. For example we took inspiration from the Raising the Floor Solutions Masterlist as we found out that classical categorization schemes given e.g. by common standards were not fit for the users we had in mind building the Developer Space.

While providing solutions in terms of specific needs was clearly a given criterion for categorization, we looked at many other functional development sites and research on adoption of open source software by developers and decision makers to understand better useful criteria for our prototype. In the end we decided to rely on our concrete target group when building the prototype, as they are our first users, without whom bootstrapping such an effort that includes a strong community will not be possible.

Page 22: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

22

Figure 11: Important factors when selecting Building Blocks (source: D402.2)

Both the searchable prototype and the documented content delivered within this work particularly consider the output by surveys and formative evaluations on wireframes conducted by the eval team during the last year (see Figure 11). E.g. the faceted search mechanism allows direct filtering by License (see Figure 12)

Figure 12: Search facets of the prototype

Page 23: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

23

An important factor for the Developer Space will be that users (i.e. implementers) find useful tools and that developers can offer their tools in a fashion that they are found specifically by stakeholders in the accessibility domain. Therefore it is wise to look particularly at categorization schemes. While this might be not so important for mainstream software developers looking for concrete functionality, many non-classical developer types we want to address, like AT product developers, healthcare professionals or relatives have a domain specific view. Furthermore procurement might be a driving factor for accessibility improvements and the need for adoption of components, so that alignment with existing schemes might prove as an opportunity in some cases.

The final editing interface of the Building Block repository that we have built on top of Drupal 7 is shown in Figure 13. It provides an easy to use web interface that allows developers to add content. The categorization is done via a tagging interface whose categories were derived from our research with users and whose terms were bootstrapped from existing sources like the solution master list or other sites. Following an inclusive approach the tagging interface, however, allows the user to specify new categories, if needed. It will be one challenge to curate and merge those categories in the future, however, already during the bootstrapping this method gave us good feedback e.g. on missing dimensions (categories).

Page 24: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

24

Figure 13: Editing interface for the repository

Figure 14 shows how a contributed component looks like in the listing. The categories allow users to easily discover similar content or to filter searches (like in Figure 15) based on the given criteria.

Page 25: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

25

Figure 14: Example Listing

Important elements of the current listing structure are:

• Quick access to examples (Getting started) • Quick access to further documentation (Read more) • A short overview description with an image • Filtering/Comparison based on relevant categories for implementers • Contact information • Access to up to date code • Commenting features (currently disabled until moderation possible) • Ability to add external video or demo content • Use of crowd-source tagging with a fixed initial shallow category (tag type) structure

and initial tags

Important additional features to enable the use cases in SP1 (not yet available) will be added through the Quality Infrastructure, the overall gamification elements and the linking with other parts of the GPII Developer Space.

Page 26: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

26

Figure 15: Search results with faceted browsing capabilities

1.3.3 Considerations for the second prototype of the repository *

While teams worked on improving their individual components, a large effort was done to actually include the Building Block Listing into a sustainable version hosted by Raising the Floor International. Many of the individual developments were redone in a way that the site shares a common technical Basis with other sites of the GPII like the Unified Listing. This is important both in terms of accessibility but also maintainability of the site.

Furthermore it will allow us now to include many other parts of the architecture (see section 1.4) on a common technical basis. This effort took much longer than expected, but has brought us much closer towards a sustainable operation, which is one of the main objectives. Other than adding many more components (we have reached 100 entries), we also refined the structure and the information architecture to fit the global developments within the GPII Developerspace and to match the feedback from the earlier evaluation rounds with implementers.

Page 27: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

27

1.3.3.1 Reduction categorization schemes that conflict with quality *

Already while preparing the first prototype of the Building Block Repository we discovered, that is was very hard for technical teams to describe their components in a way that make sense for general structuring within the accessibility space.

In the first prototype we offered three domain specific categorization schemes, we furthermore opened those categorization schemes to user defined tags allowing the categorization to be refined over time. Categorization was done by technology category (e.g. Web Components, Computer Vision Components or Smart Home Components), we offered categorization by disabilities and last by typical solution strategies that were established within the RtFI Masterlist.

While the categories seemed all helpful to users, the quality of categorization of user provided content was inferior with some people tagging with many other with few other with inconsistent categories. Thus we reduced the tagging to predefined major disability categories (deaf/hard of hearing, blind/low vision, cognitive/learning, physical). We further cleaned up the technological categories, suggesting the users common tags.

1.3.3.2 Cognitive Funneling with good READMEs *

We found that for many contributed components, either the description in the DSpace or the linked description in the repositories were not appealing to potential users providing either to short or too diverse information in different places. Also the descriptions followed different strategies. Descriptions in different places were further not updated. Thus we changed the strategy completely by asking the SP2 teams to provide high quality READMEs in the original repositories, further linking the GPII Developer Space from the repositories on Github.

Now every single component has

a) a public code repository (on github, bitbucket,....)

b) a LICENSE file within this repository that is compatible with the project (

c) a README document within this repository that follows a common style directed towards developers

d) the README contain a text with acknowledgement of the funding and links to the GPII developer space.

The structure of each README should now follow established principles (see https://github.com/noffle/art-of-readme ) that are:

Page 28: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

28

• One-liner explaining the purpose of the module • Necessary background context & links • Potentially unfamiliar terms link to informative sources • Clear, runnable example of usage • Installation instructions • Extensive API documentation • Performs cognitive funneling • Caveats and limitations mentioned up-front • Doesn't rely on images to relay critical information • License

Particularly the principle of “cognitive funneling” is important to our work as developers expect information about code components to follow common structures and to contain certain established aspects. The above cited “art of readme” resource that we used as a basis provides a structure, based on the analysis of “hundreds of modules” by the author.

A one liner also helps us to have much more concise search results.

1.3.4 Planned final refinements the release *

As this report was written another round of evaluation was conducted with the second prototype. While not results have been fully analyzed within an evaluation report and understood by the development team, we are want to already report on some direction we are foreseeing as a reaction to the feedback provided within user testing.

1.3.4.1 Remove categorizations of Code *

Based on early evaluations and user interviews we implemented purposefully two strategies for browsing the site: Searching the site and browsing the site. While we are trying to explain categories to the users. One major problem is that many terms like services, resources have specific connotations in different user groups, while other choices of terms like code components, framework and tools seem arbitrary to the users. When people browse the site, we discover a problem that can be attributed remotely to the effect of “premature cognitive commitment”. People browse the top level categories with assumptions what they find underneath each entry of the information architecture. Once they gather enough data, i.e. they found out that this not the category they thought it was, they have great difficulties to adjust this understanding of the information architecture on the GPII Developer Space. A further problem is that the current information architecture is not helping people to recover from wrong browsing choices made: if I search for something in the wrong subcategory, I would have to start from the beginning to recover.

Page 29: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

29

In terms of the Building Block repository the most sensible thing to do is to actually remove subtle differences between Code Components, Tools and Frameworks, and allow use facetted searching directly on the top category. Another step would be to display search search previews (results from other categories) to allow users to recover from early choices made on wrong assumptions.

1.3.4.2 Lowering the cost for contribution *

The latest user tests exposed that the average time to fill out a component description was 6 minutes although it involved only copy & pasting information and setting some categories. We already removed many difficult choices and provided easy interfaces for choosing commonly used tags. Yet still the time needed to add your first component was long with relatively many users aborting the task.

Adding a code component listing to the site needs to be as easy as dropping a link in our opinion or no user contribution will happen. Particularly as nearly all information is redundant to the information also provided via github. Also feedback from internal project teams indicate that they will most probably not maintain the information as regularly as they would do on github. Without any integration to automatically gather most relevant information directly from repositories like github we see no sustainable way to keep information about the component up to date and high quality.

1.3.4.3 Added value through expert categorization *

Although we have restricted the number of possible choices, we are seeing nearly no way that component developers can themselves provide information on potential uses of their component towards providing accessibility solutions in an easy fashion.

We now have gained the practical experience, that if we want to list components particularly related to certain need or addressing a certain technical solution, those categorizations need to be highly curated by experts. While we don’t want to limit AT expert to find innovative new solutions, especially people new to the AT domain will rely on curated lists. Thus we will in the final prototype try to implement a workflow, that a) updates technical information automatically b) allows experts to add accessibility related information to a component, that provide guaranteed added over the original github listing

1.3.4.4 Sort by quality metrics *

The quality infrastructure metrics have until this point only been implemented on the single component pages. However we need metrics to actually support the sort and filter components, as they provide at least one indicator for relevance towards implementer choices.

Page 30: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

30

1.3.4.5 Incentives to contribute *

While we are trying to lower the needs for external contribution to make the site sustainable, we need users to actively contribute by reviewing and reporting or by volunteering to contribute to open source. The integration of the feedback system and the gamification server will be critical next steps as those components now have been finalized.

1.4 Architectural Integration of the Building Blocks Repository *

During the creation of the first prototype of the GPII Developer Space the architecture work package proceeded with the integration of the overall architecture. The architectural components were presented in the report on the “Specification of Architecture, Security, Payment Infrastructure, and Developer Space”. During the second phase of developments we started implementing relevant parts of this architecture. Not all of those implementation have been released to the testing versions of the GPII Developer Space as development of some components is still in progress in other workpackages, but will be released step by step as technical validation is finished.

1.4.1 Integration with the P4A Quality Infrastructure and external Repositories and curated component listings *

As mentioned before it is highly critical for a successful GPII Developer Space to provide both a broad range of components also from outside the project, to give access particularly to high quality resources and to provide up to date information.

Purely based on manual curation of the content this is infeasible. Therefore curation of the Building Block Repository needs to be supported by automated tools that allow users to judge the software maturity and the maintenance activity of software components. As accessibility targeted software components are highly interaction oriented and may require special operating systems or hardware the architecture foresees a GPII Quality Infrastructure. This Infrastructure is an added value that is available to all code components that are listed via the GPII Developer Space within the Building Block Repository.

As detailed in the deliverable and shown in the figure below the integration is done via the code repositories, that also serve as identifiers to the GPII Developer Space. Those metrics and status reports are collected on different levels of detail (only commit messages for all, build reports for some) for all projects.

Page 31: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

31

Figure 16: The Search (planned) and Web (done) Components of the QI are integrated into Repository

The following screenshots show the integration of github-based quality metrics (here some Development Activity Indicators) for a single Building Block as it is hosted on github and integrated using the Infusion Framework into the Drupal-based Developer Space Prototype. This part was not yet tested with developers but will be part of the next round of testing.

Page 32: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

32

Figure 17: Integration of the QI Web Component displaying quality metrics

For the final release we are planning an even deeper integration with external repositories through the quality infrastructure. As detailed in the previous section we need to further cut down the cost of contribution. The time (average 6 minutes) needed to actually contribute content to the side is much too high for most project owners or other users. We have seen ourselves that without manual intervention the information displayed on the site quickly gets outdated. Therefore we are planning to build on the same APIs the Quality Infrastructure is building on to pull information from the source repositories as detailed above.

1.4.2 Integration with the P4A Feedback Mechanisms and Gamification *

One of the most important added values of another listing of components is additional accessibility related information. This information is on one hand provided by curation by expert. Another important source will be user ratings and reviews. During the last project period the appropriate module was integrated with the component repository and allows users to provide an additional quality indicator to single components via an integrated rating widget that is also used within the Unified Listing.

Page 33: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

33

Once the developer facing side of the feedback system is integrated it provides an important feedback channel from the end-user via the solution implementer to the component developer. The GPII Developer Space will be the entry point to this two-sided interaction.

Figure 18: Feedback component as used as part of the repoisitory

Another component that is close to be fully integrated into the Building Blocks Repository is the Gamification server “Bubbles”. As part of the GPII Infrastructure the Gamification Server allow to specify quests with rewards for improving or extending accessibility based components. This activity can be seen complementary to the Quality Infrastructure and the Feedback System as it encourages developers to actively contribute to component development.

Figure 19: Gamification quests as displayed for each component

“Karma” earned via the accepting quests will displayed as part of the user profile of the developer space incentivizing users to active contributions to the community.

Page 34: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

34

Figure 20: Gamified user profile with badge

1.4.3 Integration with the P4A Security Component and other components of P4A *

One of the most important preconditions for integrating the Unified Listing with the Developer Space is a common single-signon-provider that manages identities across those sites while keeping personal information centrally managed.

Figure 21: Single-sign-on Service allows user journeys between sites

1.4.4 Integration with the Development Environments *

One goal that is yet to be achieved is easier integration of the Building Blocks repository with the development process and easier ways to try out technology immediately. While we are

Page 35: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

35

now providing demo and tutorial links for many components (video training courses are also in preparation), there is still often barriers in trying out technology without much effort.

The WebACS that was developed within the Frameworks and Tools (WP203) work actually will enable us to preview models within the web browser. In combination with the installers generated within this work package and the REST enabled runtime it enables users also to quickly try out models on their own hardware (if available).

In future releases we are evaluating sharing of models similar to popular web application like jsfiddle.net that allow users to share and modify models over the website. This activity will also build upon the Asterics Model Wizard which is planned to be delivered in month 40 of the project.

Page 36: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

36

2 List of contributed Building Blocks by Task

In this chapter we report the list of different Building Blocks that are contained in the repository. The structuring does not follow the categories that we used in the online prototype, but reflects initial structure that we used in the Description of Work for the funded Prosperity4All project. The subsections refer to the task number within the Workpackage 202 (e.g. section 2.4 refers to Task 202.4). While large parts are available online we added dedicated information that shows the progress made within the project. This information is shown as boxes in section 2.2 onwards. Before we start explaining the different components, section 2.1 (Task 202.1) particularly also covers the collection of (project) external Building Blocks by either partners of Prosperity4All or external developers that are part of a growing open source collection.

This version of the report additionally contains section 2.6 and section 2.2.3 listing the components still missing in the last version of this report (namely the open source version of the media transformation infrastructure and the adaptive web components).

2.1 External Building Blocks and online collection (FHTW)

One important task of the WP202 activities is to conduct an extensive research on already available open source modules, libraries and components by 3rd party developers. The description of these external building blocks and their integration into the first version of the Developer Space Repository helps to establish a comprehensive collection of AT-related resources for developers, in addition to the building blocks which are contributed by P4All consortium members in SP2/WP202. In subsequent improvements of the Developer Space Repository infrastructure, database federation and the automated integration of external sources will be supported so that further (manual) addition of external Building Blocks will become less important.

By the time of project month 22, a first set of external building blocks was collected (mainly by FHTW and KIT) and added to the D-Space repository prototype. These modules contain 3rd-party libraries, coding demos, hardware designs, and build instructions as well as stand-alone software tools for applications in various fields of accessible design and assistive technology. For the stand-alone tools it will be decided at a later stage of the implementations if those tools should be classified as “products” and moved to the Unified Listing repository.

Between project month 34 and 36 we added a high number of external modules reaching 100 listed items. However, we noted that the current way of adding content is not scalable

Page 37: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

37

enough to cover all areas of accessibility. We, however, see that users are starting to acknowledge the richness of in

The following tables gives a short overview on some of the collected resources and building blocks:

Table 1 External Building Blocks for Input/Output and special interfaces

Resource / ext. Building Block

Description Link

Input Stick Android- and iOS compatible USB receiver supports wireless keyboard, mouse, game controller HID emulation

http://inputstick.com/index.php/en/

MaKey Makey

physical computing, alternative input devices http://makeymakey.com/

SmartGattLib Bluetooth LE Java library that simplifies the work with Bluetooth SMART

https://github.com/movisens/SmartGattLib

V-USB SW-defined USB, many applications for alternative HID input devices

https://www.obdev.at/products/vusb/prjhid.html

Table 2 External Computer Vision Building Blocks

Resource / ext. Building Block

Description Link

EyeWriter open source gaze tracking HW/SW http://eyewriter.org/developer/

Headtrackr Javascript library for real-time face tracking and head tracking

https://github.com/auduno/headtrackr/

tracking.js Computer vision algorithms for web applications in Html5/javascript

http://trackingjs.com/

Table 3 External Blindness/ low vision / speech creation Building Blocks

Resource / ext. Building Block

Description Link

Page 38: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

38

DAISY pipeline digital content migration to various formats; production and distribution of DAISY Digital Talking Books

http://www.daisy.org/pipeline2/overview

Describler accessible and reusable SVG images

https://github.com/shepazu/describlerhttps://github.com/shepazu/describler

ESpeak

Speech Synthesizer http://espeak.sourceforge.net/

LibLois Open-source braille translator, back-translator and formatter

http://liblouis.org/

Speak.js Speech Synthesizer javascript library

https://github.com/kripken/speak.js

SVG Sonifier Sonification of SVG charts and graphs via Web Audio and Speech APIs

http://schepers.cc/invisible-visualization

Table 4 JavaScript / HTML5 / Web related:

Resource / ext. Building Block

Description Link

AFB Accessible, embedded video player with HTML5 controls

http://www.afb.org/info/programs-and-services/technology-evaluation/creating-accessible-websites/download-afbs-accessible-html5-video-player/1235

User Interface Options

component allows users to transform the presentation of the user interface and content resources so that they are personalized to the individual user's needs

https://github.com/fluid-project/infusionhttps://github.com/fluid-project/infusion

Infusion Table of Contents component

component examines a page for HTML heading elements, and generates and formats a list of links to headings that can be used to navigate the page.

https://github.com/fluid-project/infusionhttps://github.com/fluid-project/infusion

Infusion Text-to-Speech component

javascript wrapper around the SpeechSynthesis Interface from the Web Speech API. Web Speech API.

https://github.com/fluid-project/infusionhttps://github.com/fluid-project/infusion

First Discovery Tool

Preferences Editor designed to be an entry point for users who are new to customizing a user interface to

https://github.com/GPII/first-discovery

Page 39: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

39

match their own needs and preference

Table 5: Remote controll Building Blocks

Resource / ext. Building Block

Description Link

jQuery URC uses jQuery to simplify the access to the official URC webclient library.

http://sourceforge.net/projects/traceurcsdk/files/Webclient-Generic-Client/3.1/http://sourceforge.net/projects/traceurcsdk/files/Webclient-Generic-Client/3.1/

JavaScript URC

binding to UCH-provided sockets, based on the URC-HTTP 2.0 specification

http://www.openurc.org/tools/friedolinfoerder-jquery.urc.js-1.0beta/jquery.urc.js

Table 6 Other external Building Blocks

Resource / ext. Building Block

Description Link

AutoHotkey scripting language for OS automation https://autohotkey.com/

AMCharts Accessibility enhancements for a very versatile chart library http://paypal.github.io/amcharts-accessibility-plugin

Bootstrap extension for the Bootstrap 3 web development framework, adds keyboard and screen reader support

http://paypal.github.io/bootstrap-accessibility-plugin/

Docverter Service Convert documents written in HTML, Markdown, or LaTeX to PDF, Docx, RTF or ePub via HTTP API

http://www.docverter.com

Readium JS library for EPUB3 publications http://readium.github.io/index.html

Page 40: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

40

2.2 AT Specific I/O Modules (P4A T202.2) In Task 202.2, specific hardware and software modules by P4ll partners are refined (or newly created) and integrated into the P4All infrastructure. In particular, these modules enable special input- and output capabilities for building alternative Human-Computer Interface (HCI) solutions and accessible interaction strategies for novel (or existing) products, including microcontroller-based input (measuring sensors, processing switch input), output (controlling external actuators) or camera-based input solutions (computer vision algorithms for face tracking or feature extraction from live camera images). The following sections give an overview on the state of these components:

2.2.1 AsTeRICS AT Modules (FHTW)

In course of the AsTeRICS project (FP-7 STREP, 2010-2013), a flexible system for graphical design of Assistive Technology solutions has been developed, which includes a JAVA/OSGi-based middleware / runtime system, a graphical editor for the design of assistive solutions (so-called “AsTeRICS models”) and more than 150 sensing-, processing and actuator-plugins which can be combined and parameterised using the graphical editor.

In the first year of the P4ALL project, strategies for a flexible integration of AsTeRICS-based solutions into various application scenarios have been evaluated and implemented by consortium member FHTW. Two AsTeRICS-based solutions were already applied in course of SP3-implementations by consortium members LIFEtool and GuadalInfo.

Although it was initially planned to completely extract certain useful plugins from the AsTeRICS framework in course of the WP202 tasks (respectively remove the bindings of these plugins to the middleware and provide them as plain JAVA/C++ source code), the discussions with the SP3-implementers revealed that it would be more useful for them to provide a customizable, shrinked version of AsTeRICS (including the middleware) which maintains the flexibility of the graphical design and adaptation of the solution, but significantly reduces the needed system resources for its deployment and adds certain communication channels to transfer live sensor data and control signals from/to the AsTeRICS model to/from the implementer’s application. Thus, the AsTeRICS runtime environment could run in parallel with the implementer’s application and expose additional GUI components only if needed.

This concept was realized during project months 12-20 and turned out to be successful and very flexible. AsTeRICS building blocks were implemented by LIFEtool (for their Windows-based learning software “FlashWords”) and by GuadalInfo (for their Linux-based computer terminals) to add camera-based cursor control.

To facilitate the process of building shrinked-down AsTeRICS deployments to different target platforms which only contain the necessary plugins / building blocks and services, FHTW

Page 41: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

41

developed the AsTeRICS Packaging Environment (APE) and a websocket interface for data exchange with running AsTeRICS models, and a strategy for dealing with the various (and often concurring) licenses involved.

AsTeRICS Packaging Environment (APE)

The AsTeRICS Packaging Environment allows the selection of a set of AsTeRICS model files and creates a downstripped (minimum size) version of the AsTeRICS Runtime Environment (ARE) including plugins, configuration files and data files to execute these models. Optionally, the APE allows the creation of native installers for Windows, Linux and Mac using JavaFX packaging technology..

The created packages are runtime-only versions and don't have an AsTeRICS Configuration Suite (ACS) included. The packages are intended for developers of accessible products to embed AsTeRICS functionality into their deployment by creating small and use-case optimized packages for their platform of choice. If native installers are created for the customized packages the Java Runtime Environment (JRE) is embedded and the application is seamlessly integrated into the operating system (native launcher e.g. .exe file, icon, shortcut, ...).

In course of the project Prosperity4All, several building blocks were created as downstripped versions using APE. Check several example configurations here .

Managing Licenses of the AsTeRICS Building Blocks

Due to the many different plugins and their 3rd-party dependencies with dedicated licenses, the original license of the AsTeRICS system had to be GLPv3 (because some of the plugins contain GPL’ed code which then influences all the middleware and the whole AsTeRICS release). In the P4All project, one requirement was to select the most permissive license for a particular building block or use case - which makes it possible for a larger group of implementers (including implementers of proprietary software) to use these building blocks. In course of their P4All work, FHTW integrated a license managing feature into the APE packaging tool. For a particular use case (for example involving 5 desired building blocks) the APE tool collects all license information for the involved building blocks and puts their license information into the deployment folder, so that it becomes easy for an implementer to deduce the resulting license implications.

For all code developed in the AsTeRICS project itself (including all AsTeRICS middleware and plugin skeletons), FHTW chose a dual-licensing approach (MIT or GPL) which allows to license these source code modules for a particular deployment either with MIT (most permissive) or GPL (if other GPL code is contained in the deployment). For more information please refer to the AsTeRICS P4All Building Blocks repository: https://github.com/asterics/P4AllBuildingBlocks/blob/master/LICENSE_dual.txt

Page 42: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

42

2.2.2 Websocket Interface for AsTeRICS Building Blocks (FHTW)

The websocket building block provides a generic bidirectional data interface to a running AsTeRICS deployment, where other AsTeRICS-based building blocks perform sensing, processing or actuation tasks. Using this interface, sensor data can be forwarded to a browser or another application to visualize the data or trigger actions. Vice versa an application can send data into the AsTeRICS system - for example to trigger actuators like a home control appliance.

Addr. Special Needs: General Inclusion/Assistance

Technology Type: Prototyping

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: diverse (depending on application)

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/NetworkIO/

2.2.3 RoboBraille.Web.API (Sensus) *

RoboBraille is a web service capable of automatically transforming documents into a variety of alternate formats for the visually and reading impaired. RoboBraille has been available as web application for many years. Now, RoboBraille is available as a programmable Web Api. The objective is to provide developers with the basic support for creating their own web server implementation of the API. The Web API infrastructure provides the backbone on which the RoboBraille conversions take place. The RoboBraille Web Api was developed at Sensus ApS. Sensus is a research-based consultancy organisation specialising in accessibility, inclusion, information technology and disability.

The RoboBraille Web Api project is now available as an open source Web Api application, however the technologies used by the application are each subject to their own licensing, terms and conditions.

Page 43: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

43

Addr. Special Needs: Blindness,Low Vision,Cognitive

Technology Type: Media Transformation

License: Apache Licence 2.0

Used/Supported Tech.: C#, DAISY, EPUB

Addr. Tech. Solution: Alternative formats

Source Repository: https://github.com/sensusaps/RoboBraille.Web.API

2.3 Generic Multimodal Interaction Modules (P4A T202.3) In course of task T202.3, several hardware and software modules have been further developed or refined for use within the P4All ecosystem by the responsible P4All partner organisations. These modules serve various kinds of Human-Computer Interaction tasks and/or allow interfacing of multimodal sensor data for application in assistive applications. Specifically, the integrated modules include microcontroller platforms for interfacing sensors or actuators to desired computing systems, camera input modules which use computer vision algorithms for alternative computer input and haptic/touch I/O modules for adding haptic feedback to various user interface components.

2.3.1 Arduino UNO: Microcontroller Interface (I/O Transducer Modules) (FHTW)

The Arduino component provides an interface to the popular Arduino Uno microcontroller and makes analog inputs and digital inputs and outputs available for desired applications. The Arduino UNO is one of the most popular boards of the constantly growing Arduino ecosystem. It provides 13 digital input/output pins and 6 ADC-channels with 10 bit resolution and 3 PWM channels for D/A conversion.

Page 44: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

44

Addr. Special Needs: Physical, General Inclusion/Assistance

Technology Type: Microcontrollers

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: C, C++, Java, AsTeRICS

Addr. Tech. Solution: Interfacing of sensors / actuators, environmental control

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/Microcontroller/Arduino

2.3.2 Universal HID Actuator: Mouse/Keyboard/Joystick Emulation (FHTW)

The Universal HID Actuator is a hardware module which emulates Mouse, Keyboard and Joystick HID devices on a target computer. It receives the control commands via a wireless Bluetooth link. The HID Actuator can be used to enable mouse cursor control, keyboard input or game console control via a personalised input

setup - no extra driver installation on the target computer is needed

Addr. Special Needs: Physical

Technology Type: Input Emulation

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: C, Java, AsTeRICS

Addr. Tech. Solution: Input Device Emulation

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/Microcontroller/UniversalHIDActuator

2.3.3 Camera Input Module: FacetrackerLK, XFacetrackerLK: (FHTW)

Page 45: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

45

The FacetrackerLK component uses a standard webcam or external camera and provides movement information of facial features from a live camera video. The estimated relative movement of a user's nose and chin are provided in x- and y-direction.

The FacetrackerLK component builds upon OpenCV and uses a combination of the HaarCascade and the Lukas Kanade Optical Flow Tracking Algorithms to provide

efficient and accurate tracking also on low performance platforms. A re-calibration of the face position is initiated whenever excessive drifting of the feature spots is detected

Addr. Special Needs: Physical

Technology Type: Computer Vision

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: OpenCV, C++, Java, AsTeRICS, Windows

Addr. Tech. Solution: Alternative User Interface

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/CameraInput/FacetrackerLK-windows

2.3.4 P4A Haptic Module (CERTH)

P4A WebHapticModule aims at simplifying the whole process of adding haptic feedback to a web application, towards improving web content’s accessibility and also enhancing end-users’ cognitive capabilities. It also makes web content exploration more entertaining.

Page 46: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

46

Addr. Special Needs: Low Vision

Technology Type: Web Technology

License: BSD

Used/Supported Tech.: C++, JNI, Java, Windows

Addr. Tech. Solution: Alternative User Interface

Source Repository: https://github.com/P4ALLcerthiti/WebHapticModule

2.3.5 P4A Android Vibration Module (CERTH)

P4A AndroidVibrationModule is an Android Library that offers a mechanism for easily adding vibration feedback to Android applications. Even if it is based on the native Android vibration framework, however, it offers greater simplicity compared to the native framework.

Addr. Special Needs: Low Vision, Cognitive

Technology Type: Android App

License: BSD

Used/Supported Tech.: Android

Addr. Tech. Solution: Alternative User Interface

Source Repository: https://github.com/P4ALLcerthiti/AndroidVibrationModule

2.4 Smart Device and Environment Modules (P4A T202.4) Integration with smart environments offers new business opportunities for AT providers with regard to the internet of things and ambient assisted living. Building blocks that enable real world interaction both regarding control and discovery were developed as part of the project. One goal is to integrate also with standards like the Universal Remote Console (URC) technology (ISO/IEC 24752) and its technical standards for implementation (published by the openURC Alliance). A special focus was put on the inclusive aspect of smart environments that allow spontaneous interactions of users with that environment, i.e. the fact that anything that can be controlled by user standing in front of a switch should be able to be controlled by anyone at a similar position with the same ease.

2.4.1 EnOcean: Smart Home Integration (FHTW)

Page 47: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

47

EnOcean is an energy-harvesting wireless sensor network widely used in home- and building automation tasks. As KNX, enOcean provides many different sensors and actuators for smart home control and management. This component utilizes the Priscilla java library for the EnOcean

implementation. The EnOcean plugin provides an interface to the EnOcean sensors over an USB stick (EnOcean USB300) or an IP gateway.

Addr. Special Needs: Physical

Technology Type: Smart Homes and Building Automation

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: Accessible Buildings

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/SmartHomeIntegration/enOcean

KNX: Smart Home Integration (FHTW)

The KNX standard is one of the major European standards for building automation. Many different sensors and actuators are available from various manufacturers including lighting control, heating, ventilation and air conditioning, automated blends or door actuators.

Addr. Special Needs: Physical

Technology Type: Smart Homes and Building Automation

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: Accessible Buildings

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/SmartHomeIntegration/KNX

Page 48: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

48

2.4.2 Point and Control: Spatial Interaction for Remote Control (KIT)

This module enables the user to place and retrieve URLs in the environment by simple pointing gestures. This is realized with the use of the Microsoft Kinect depth camera, which can track multiple users in the room.

Addr. Special Needs: Literacy

Technology Type: Remote Controlling, Spatial Interaction, Physical interface

License: MIT

Used/Supported Tech.: C#, Javascript

Addr. Tech. Solution: Simplifying, Adjustable or Alternative Input Devices

Source Repository: https://github.com/teco-kit/PointAndControl

2.4.3 IndianaJS: Spatially aware Web of Things (KIT)

IndianaJS is a spatial location and interaction library for Web-of-Things-enabled web sites. It scrapes web sites annotated with relative position information and transforms locations relative to the user. E.g. makes it easy to add screen-reader friendly dynamic directions to real-world related content.

Addr. Special Needs: Blind, Low Vision

Technology Type: Web Technology, Remote Controlling

License: MIT

Used/Supported Tech.: Javascript, HTML5

Addr. Tech. Solution: Remote User Interface, Real-Time Navigation

Source Repository: https://github.com/indianajs/indiana-js

Page 49: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

49

2.4.4 Context-WiFi Login (KIT)

This building block contains source code for different tag and context based authentication schemes for open Wi-Fi access for so-called captive portals.

It uses (polymer) web components to provide context-adaptive login mechanisms that can be added post-hoc to many WiFi installations where a basic level of access control is needed.

This includes access to accessible smart home installations in your home or providing access to your company guest network without inaccessible password schemes that use paper.

Addr. Special Needs: Blind, Low Vision

Technology Type: Web Technology

License: BSD

Used/Supported Tech.: Web Components, Polymer, HTML5, Javascript

Addr. Tech. Solution: Alternative User Interface

Source Repository: https://github.com/teco-kit/UsableWi-FiAccess

2.4.4.1 Universal Remote Console Socket Templates

An important prerequisite for the success of sockets is that they are as stable as possible. This should be avoided as the change of a target’s socket makes it also necessary to adjust all related user interfaces. Hence, the OpenURC Alliance provides a set of standardized master sockets that can be refined and used for any specific target.

A master socket should be applicable to a whole class of targets. In order to equip any target in that class of devices with a socket description, developers can choose from a set of master sockets. If it is intended to describe only the most common, functionality of a device or service that is typical for a target class, sockets can be directly used without any adjustments. If a developer wants to add additional features he can still use a master socket to describe the most common features and extend it by device specific aspects.

This reuse of master sockets reduces the risk of failures, reduces the effort of creating new sockets and supports the grouping of devices in classes.

Page 50: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

50

Wöhlke Web Electricity Socket Template 1.0

This document describes the various URC master documents for a “Wöhlke Websteckdose”. The Electricity Socket developed by Wöhlke is a via HTTP controllable electricity socket with an integrated temperature sensor. Based on these documents personalized user interfaces can be developed in order to control such a device. All related documents were developed by HDM during the first year of P4a.

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/TPL/WoehlkeWebsteckdose-1.0-yyyymmdd/

Plant Sensor Templates 1.0

This document describes the various URC master documents for any plant sensor. All documents can either be used directly or can serve as templates for more specialized devices. Based on the abstract definitions of the plant sensor personalized user interfaces can be developed.

All related documents were developed by HDM during the second year of P4a.

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://openurc.org/TPL/MasterTemplates/plantsensor-1.0-yyyymmdd/

NetAtmo Templates 1.0

This document describes the various URC master documents for the NetAtmo weather station. Based on these documents personalized user interfaces can be developed. The NetAtmo weather station is a device that can record different in- and outdoor values about the current weather situation. More information can be found at https://www.netatmo.com/en-US. All related documents were developed by HDM during the second year of P4a.

Page 51: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

51

Coloured light bulb Templates 1.0

This document describes the various URC master documents for a coloured light bulb. Based on these documents personalized user interfaces can be developed for controlling any lighting system. This includes colour, brightness and saturation. All related documents were developed by HDM during the second year of P4a.

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://openurc.org/TPL/MasterTemplates/ColouredLightBulb-1.0-yyyymmdd/

DVD Player Templates 1.0

This document describes the URC master documents for any DVD Player. All URC Master documents are on a generic level andan can be used by user interface developers or by Vendors to make their device controllable via the URC framework. During the first to years of the P4a project, HDM provided assistance and guidance for writing and standardizing all related documents.

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/TPL/dvd-player-1.0-20170120/

TV set 1.0

This document describes the various URC master documents for a TV-set. They can be used as template or the final URC files for any controllable TV-set.

Page 52: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

52

During the first to years of the P4a project, HDM provided assistance and guidance for writing and standardizing all related documents. Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/TPL/tv-set-1.0-20160915/

2.5 Real-Time User Monitoring Modules (P4A T202.5) Detecting the users need automatically is one the important prerequisite for adaptive, inclusive interfaces. Building Blocks developed within the project allow developers to gain more information about the current state of a user and to dynamically react to context beyond static preference sets. Different components provide a technical basis for integration of a diverse set of inputs beyond explicit interaction technology. The goal is the integration with the run-time components developed in WP203 and to provide a basis for advanced contextual user interface adaption. Current building blocks provide advanced processing facilities employing advanced processing and cognitive system technology as a first step.

2.5.1 P4A SensorAPI (KIT)

The API provides web developers an easy and consistent way to handle devices with sensors for multimodal inputs.

This framework provides unified APIs to access sensor that integrate smartphone build-in sensors, external BT-LE Sensors (as already used for fitness tracking, health applications), environment sensors via Wi-Fi, affect sensing sensors, virtual sensors implemented using machine learners or e.g. model driven processing tools like Asterics in a unified way via web interfaces (such as WebRTC, WebSockets) via a driver abstraction.

Page 53: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

53

Addr. Special Needs: General Inclusion/Assistance

Technology Type: Sensor and Input Abstraction

License: Apache License 2.0

Used/Supported Tech.: Javascript, Cordova

Addr. Tech. Solution: Adjustable or Alternative Input Devices

Source Repository: https://github.com/teco-kit/p4a-sensorAPI

2.5.2 FallDetectionModule (CERTH)

P4A FallDetectionModule is module that detects falls (and/or potentially other human locomotion activities) based on the activity-related recordings of accelerometers. The core algorithm of the module has been built upon a neuro-fuzzy approach.

Addr. Special Needs: General Inclusion/Assistance

Technology Type: Desktop

License: BSD

Used/Supported Tech.: C++, Windows

Addr. Tech. Solution: Interfacing of sensors / actuators

Source Repository: https://github.com/P4ALLcerthiti/P4ALL_FallDetection

2.5.3 P4ALL Affect Sensing Module (CERTH)

P4ALL Affect Sensing Module offers a way to detect stress of individuals based on the features extracted by emphatical sensors. A module that enables stress detection through the data produced by the ProComp5 Infiniti Hardware paired with the SA9309M skin conductance sensor and the EKG™ Sensor - T9306M .

Page 54: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

54

Addr. Special Needs: General Inclusion/Assistance

Technology Type: Desktop

License: BSD

Used/Supported Tech.: C++, Windows

Addr. Tech. Solution: Interfacing of sensors / actuators

Source Repository: https://github.com/P4ALLcerthiti/P4ALL-Affect-Sensing-Module

2.5.4 Social Network Interaction Module (CERTH)

P4A SocialNetworkIteractionModule is a module that performs anomaly detection and root cause analysis. Specifically, this procedure takes place with the use of K-partite graph.

Main features

The main features of P4A SocialNetworkIteraction include the following:

• Two sets of test data: The first set is a simple example and the second set is an example with real data from twitter where the followers and the following people from random users are given.

• Selection of parameters for k-partite graph. Default values are also given. • Level positions and the number of connections of each node are given as a result.

Page 55: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

55

2.5.5 jActivity (KIT)

The module adds the possibility to classify the user activity context during the browsing based on the acceleration of the smart phone and leverages supervised machine learning to create virtual sensors.

Addr. Special Needs: General Inclusion/Assistance, Vision, Cognitive

Technology Type: Activity Recognition, Virtual Sensors

License: MIT

Used/Supported Tech.: HTML5, Javascript, OpenCPU, R

Addr. Tech. Solution: Interfacing of sensors / actuators

Source Repository: https://github.com/teco-kit/jActivity/

2.5.6 Open BCI: Bioelectric Signal Acquisition and Processing (FHTW)

The Open BCI project provides open source hardware and software for bioelectric signal acquisition systems (biosignal amplifiers). The OpenBCI hardware consists of an 8-channel

bioelectric signal amplifier board with TI ADS1299 24 bit EEG front end, SD card slot and Bluetooth Low Energy (BLE) wireless link. It also includes an ST LIS3DH accelerometer.

Addr. Special Needs: Physical

Technology Type: Bioelectric Signal Acquisition and Processing

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: Adjustable or Alternative Input Devices

Source Repository: https://github.com/asterics/P4AllBuildingBlocks/tree/master/BioelectricSignal/Op...

Page 56: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

56

2.6 Adaptive Web (P4A Task 202.6) *

2.6.1 Adaptive Web Components (HDM/UStutt) *

Web Components are a set of features currently being added by the W3C to the HTML and DOM specifications that allow for the creation of reusable widgets or components in web documents and web applications. The intention behind them is to bring component-based software engineering to the World Wide Web. Adaptive Web Components (AWC) adds the ability to use adaptive components, which consider the need and preference set of a user to improve the accessibility, usability and user experience.

Addr. Special Needs: General

Technology Type: Web Technology

License: Apache License 2.0

Used/Supported Tech.: Web Components, HTML5, Javascript, CSS

Addr. Tech. Solution: Web Technologies

Source Repository: https://github.com/REMEXLabs/AWC

2.6.2 CSS Context Query (UStutt) *

Context Query provides an extension of the widely used CSS Media Queries. To be able to adjust a web application to certain devices, the @media rule controls the styles via type and feature, mostly screen and width/height respectively. The @context rule defines the style rules for context based features like the ambient light levels near the device amongst others.

Addr. Special Needs: General

Technology Type: Web Technology

License: MIT

Used/Supported Tech.: Web Components, HTML5, Javascript, CSS

Addr. Tech. Solution: Adjustable or Alternative Input Devices

Source Repository: https://git.iao.fraunhofer.de/adaptive/contextquery

Page 57: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

57

3 Tutorials, Guidelines and Handbooks

3.1 Tutorials The following tutorials were created

3.1.1 URC Tutorials

3.1.1.1 Tutorial: Getting started with URC (HDM)

This tutorial will give you an understanding of the main components of the URC framework and the idea of exchangeable and personalized user interfaces. The main concepts of Controllers, Universal Control Hub (UCH), targets, Sockets and the Resource Server are explained in a simple scenario. For the basic example you need nothing more than a PC with Internet access.

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/TPL/WoehlkeWebsteckdose-1.0-yyyymmdd/

3.1.1.2 Tutorial: Creating Sockets with the SocketBuilder (HDM)

This tutorial explains the creation of a socket for a very simple Todo-Service that we want to control. We will create the Socket with the SocketBuilder step by step and deploy it on a local UCH instance for testing purposes. By creating the Socket the three types of Socket Elements - variables, commands and notifications - will be explained. Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/tutorials/SocketBuilder/TutorialTodoService.htm

Page 58: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

58

3.1.1 Asterics Tutorials *

3.1.1.1 AsTeRICS Plugin Development - Step by Step (FHTW) *

Explains how to write a new plugin for AsTeRICS in a step by step guide.

Addr. Special Needs: General Inclusion/Assistance

Technology Type: Prototyping

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: diverse (depending on application)

Source Repository: https://github.com/asterics/AsTeRICS/blob/master/Documentation/AsTeRICS%20Plugin%20Development.pdf

3.1.1.2 AsTeRICS Camera Mouse Model Creation - Step by Step (FHTW) *

Explains how to write a new plugin for AsTeRICS in a step by step guide. Addr. Special Needs: Physical

Technology Type: Prototyping

License: MIT / GPLv3 (dual licensed)

Used/Supported Tech.: Java, AsTeRICS

Addr. Tech. Solution: diverse (depending on application)

Source Repository: https://github.com/asterics/AsTeRICS/blob/master/Documentation/AsTeRICS_CameraMouseCreation_StepbyStep_Tutorial.pdf

3.1.1 Fluid/FLOE Tutorials

3.1.1.1 User Interface Options Tutorial This tutorial will walk you through the process of adding the Infusion User Interface Options component to your website.

Page 59: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

59

Addr. Special Needs: Low Vision, Cognitive

Technology Type: Web Technology

License: CC-BY 3.0

Used/Supported Tech.: Javascript

Addr. Tech. Solution: Personalization, Preference

Source Repository: http://docs.fluidproject.org/infusion/development/tutorial-userInterfaceOptions/UserInterfaceOptions.html

3.1.1.2 Infusion Preferences Framework Tutorial This tutorial will walk you through the process of building a Preference Editor using the Infusion Preferences Framework. Addr. Special Needs: Blind, Low Vision

Technology Type: Gaming, Web

License: CC-BY 3.0

Used/Supported Tech.: Javascript

Addr. Tech. Solution: Personalization, Preference

Source Repository: http://docs.fluidproject.org/infusion/development/tutorial-prefsFramework/CreatingAPrefsEditor.html

3.1.1.3 First Discovery Tool Tutorial The First Discovery Tool documentation provides tutorials and API information about the First Discovery Tool. Addr. Special Needs: Generic

Technology Type: Web

License: BSD

Used/Supported Tech.:

Addr. Tech. Solution:

Source Repository: https://github.com/GPII/first-discovery

Page 60: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

60

3.2 Guidelines & Handbooks

3.2.1 URC Guidelines

3.2.1.1 Design Guidelines for User Interface Sockets

URC sockets are the base on which personalized user interfaces can be build. Since sockets are the common base of all possible user interfaces they should be well designed and stable in time. These guidelines help to develop reliable and well-designed sockets so that they can be used by third parties (e.g., user interface developers)

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/tutorials/SocketDesignGuidelines.html

3.2.1.2 Making a business case with URC

This user story tells a fictive business case which shall illustrate the concepts of URC Super Sockets. Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://wiki.gpii.net/w/URC_Super_Sockets_user_story

3.2.1.3 URC Frequently Asked Questions

This is a collection of some typical questions that users and developers encounter when they are using the URC framework.

Page 61: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

61

Addr. Special Needs: Blind, Low Vision, Deaf Hard of Hearing

Technology Type: Smart Home & Building Automation

License: CC-BY 4.0

Used/Supported Tech.: URC,XML

Addr. Tech. Solution: Alternative User Interface

Source Repository: http://www.openurc.org/tutorials/FAQs.html

3.2.1 Learning Material

3.2.1.1 Accessible Standardized Testing Guide In a traditional test environment in education, if a student requires accommodations special arrangements and exceptions are required. The Accessible Standardized Testing guide on the Inclusive Learning Design Handbook aims to give strategies that provide computer-aided features which benefit all students. Addr. Special Needs: General

Technology Type: Testing

License: Creative Commons Attribution 2.5 Canada

Used/Supported Tech.:

Addr. Tech. Solution:

Source Repository: http://handbook.floeproject.org/AccessibleStandardizedTesting.html

3.2.1.2 Inclusive EPUB 3 Guide The Inclusive EPUB 3 guide within the Inclusive Learning Design Handbook (ILDH) provides practical strategies to thorny inclusivity and accessibility issues found in EPUB 3, and covers topics such as math, narration, and interactive content within EPUB 3 publications. Addr. Special Needs: Blind, Low Vision

Technology Type: Media Transformation

License: Creative Commons Attribution 2.5 Canada

Used/Supported Tech.:

Addr. Tech. Solution:

Source Repository: http://handbook.floeproject.org/InclusiveEPUB3.html

Page 62: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

62

3.2.1.3 Accessible Web Games and Interactive Simulations The Accessible Web Games and Interactive Simulations guide within the Inclusive Learning Design Handbook (ILDH) provides insights and techniques for creating accessible interactive content. Topics include orienting a screen reader user, communicating events, and cognitive load. Addr. Special Needs: Blind, Low Vision

Technology Type: Gaming, Web

License: Creative Commons Attribution 2.5 Canada

Used/Supported Tech.:

Addr. Tech. Solution:

Source Repository: http://handbook.floeproject.org/WebGamesAndSimulations.html

Page 63: Report on first prototype of Building Block Repository...D202.2 Report on first prototype of Building Block Repository 2 Abstract Three years have passed in which components have been

D202.2 Report on first prototype of Building Block Repository www.prosperity4all.eu

63

4 Conclusion and Future Work *

We are currently preparing a release of our Building Blocks repository as part of the overall GPII Developer Space. During the course of the project we made some important lesson regarding user expectations and sustainable operation. We still believe that the GPII Developer Space in general and the a Building Blocks Repository in particular are important parts of an infrastructure that is needed to enable a diverse and open ecosystem of developers.

With most of the work on components developed inside of the project finished we have a close to complete list of the technology component that have been developed inside the project. A overall Building Block repository will never be complete. Github alone lists 605 repository if one searches for “a11y” (a commonly used abbreviation for accessibility related content). Developers will expect up to date access to all this content.

Even more important the Building Block Repository needs to provide added value to implementers over web searches. This value can only come from the community and experts within this community that help to guide particularly new people through the vast amount of material. Partially we will also generate added value through specialized metrics and advanced sorting methods. But again this will also highly depend on good categorization and user contributed ratings. While integrating with many of the other architectural components, we will work in this direction.

One important value will be the stories about successful use of the components in implementations that are done in other parts of the project. Not all components will make it into commercial products, but in any case the reports on the experiences with links to the actual components will generate high value. The same is true for other very practical content like hands on tutorials and demos that link one or multiple components on the site. In combination with this type of material that is generated right now within other parts of the project the Building Block repository can generate its full value as an interconnected part of the GPII Developer Space.

Based on a solid basis that we achieved now, the developments from now on will focus on interconnecting, aggregating and interacting within a growing ecosystem. For this the most important part will be the presentation of the achievements to a larger audience.