D7a.4 Personalized News Platform prototype v3 · FP7 - 231761 Deliverable 7a.4 Page 1 of 36 NoTube...

FP7 - 231761

Deliverable 7a.4

of 36

NoTube Networks and ontologies for the transformation and unification of

broadcasting and the Internet

FP7 – 231761

D7a.4 Personalized News Platform prototype v3

Coordinators: L. Vignaroli, F. Negro, R. Del Pero (RAI)

With contributions from: Quality Assessor: Davide Palmisano

Quality Controller: Lyndon Nixon

Document Identifier: NoTube/2011/D7a.4 Class Deliverable NoTube EU-IST-2009-231761 Version: 3.02 Date: State:

Distribution: PU

FP7 - 231761

Deliverable 7a.4

of 36

EXECUTIVE SUMMARY This document presents the third implementation stage of the Personalized Semantic News use case and it is a paper companion to the software prototype D7a.4 Personalized News Platform prototype v3.

In this deliverable all issues are faced from a technical point of view rather than from a logical point of view, leaving the last one to the deliverable D7a.1 “Personalized News Services Specifications”. No relevant logical deviations have been applied during the evolution of the project.

This document describes reference scenarios for the Personalized News Platform prototype that have been implemented in this third release of the prototype. The technical implementation of the prototype is presented, with an overview of the integrated architecture, focusing on the two main sides of the foreseen NoTube architecture, the Service Provider side and the Home Ambient side.

The level of integration into this third prototype with services provided by other NoTube Work Packages is underlined, also making reference to the NoTube Integrated Platform described in D6.1b NoTube System Specifications and Architectural Design and D6.3 NoTube integrated System 2nd prototype.

A step by step GUI demo describing typical operations envisaged for the WP7a third integrated prototype, with smartphone screenshots extracted from the running demonstrator is presented as guided walkthrough of the prototype.

FP7 - 231761

Deliverable 7a.4

of 36

DOCUMENT INFORMATION

IST Project Number

FP7 - 231761 Acronym NoTube

Full Title Networks and ontologies for the transformation and unification of broadcasting and the Internet

Project URL http://www.notube.eu/ Document URL EU Project Officer Leonhard Maqua Deliverable Number 7a.4 Title Personalized

News Platform prototype, v.3

Work Package Number 7 Title TV-related Use Cases

Date of Delivery Contractual M33 Actual M33 Status final √ Nature prototype √ report □ dissemination □ Dissemination level public √ consortium □ Authors (Partner) RAI

Responsible Author Name Luca Vignaroli E-mail [email protected] Partner RAI Phone +39 0118103128

Abstract (for dissemination)

This document presents the third implementation stage of the Personalized Semantic News use case and it is a paper companion to the software prototype D7a.4 Personalized News Platform prototype v3. In this deliverable all issues are faced from a technical point of view rather than from a logical point of view, leaving the last one to the deliverable D7a.1 “Personalized News Services Specifications”. No relevant logical deviations have been applied during the evolution of the project. This document describes reference scenarios for the Personalized News Platform prototype that have been implemented in this third release of the prototype. The technical implementation of the prototype is presented, with an overview of the integrated architecture, focusing on the two main sides of the foreseen NoTube architecture, the Service Provider side and the Home Ambient side. The level of integration into this third prototype with services provided by other NoTube Work Packages is underlined, also making reference to the NoTube Integrated Platform described in D6.1b NoTube System Specifications and Architectural Design and D6.3 NoTube integrated System 2nd prototype. A step by step GUI demo describing typical operations envisaged for the WP7a third integrated prototype, with Smartphone screenshots extracted from the running demonstrator is presented as guided walkthrough of the prototype.

Keywords Personalized Semantic News, News Item, NIC, SOA, Rules, Integration

FP7 - 231761

Deliverable 7a.4

of 36

Version Log Issue Date Rev. No. Author Change 06/09/2011 1.00 RAI First draft 17/10/2011 2.00 RAI Ready for QA 19/10/2011 2.01 RAI Revisited after QA

20/10/2011 3.00 RAI Final - Revisited after QC 21/10/2011 3,01 RAI Final – Consortium table

update 24/10/2011 3,02 RAI Final – Partner contact

updated

FP7 - 231761

Deliverable 7a.4

of 36

PROJECT CONSORTIUM INFORMATION

Participants Contact

Vrije Universiteit Amsterdam

Guus Schreiber Phone: +31 20 598 7739/7718 Email: [email protected]

British Broadcasting Corporation

Libby Miller Phone: +44 787 65 65 561 Email: [email protected]

Pronetics

Marco Bruni Phone: +39 06 45472503 Email

Engin Medya Hizmetleri A.S.

Ron van der Heiden Phone: +31 6 2003 2006 Email: [email protected]

Institut fuer Rundfunktechnik GmbH

Christoph Dosch Phone: +49 89 32399 349 Email: [email protected]

Ontotext AD

Atanas Kiryakov Phone: +35 928 091 565 Email: [email protected]

Open University

Stefan Dietze Phone: +44 1908 858 217 Email: [email protected]

RAI Radiotelevisione Italiana SPA

Alberto Morello Phone: +39 011 810 31 07 Email: [email protected]

Semantic Technology Institute International

Lyndon Nixon Phone: +43 1 23 64 002 Email: [email protected]

Stoneroos B.V.

Annelies Kaptein Phone: +31 35 628 47 22 Email: annelies.kaptein@stoneroos

Thomson Video Networks Raoul Monnier Phone: +33 2 99 27 30 57 Email: [email protected]

Polymedia, SpA

Tullio Pirovano Phone: +39 02 25771 1 Email: [email protected]

KT Corporation

Myoung-Wan Koo Phone: +82 2 526 6347 Email: [email protected]

FP7 - 231761

Deliverable 7a.4

of 36

TABLE OF CONTENTS

LIST OF FIGURES ..............................................................................................................................7

LIST OF TABLES ................................................................................................................................8

LIST OF ACRONYMS AND ABBREVIATIONS .............................................................................9

1. INTRODUCTION ...........................................................................................................................10

1.1 SCOPE OF THIS DOCUMENT ..........................................................................................................10 1.2 DOCUMENT OUTLINE ...................................................................................................................10

2. PROTOTYPE ARCHITECTURE ................................................................................................11

2.1 SERVICE PROVIDER .....................................................................................................................11 2.2 HOME AMBIENT ..........................................................................................................................11

2.2.1 Back-end ............................................................................................................................................. 11 2.2.2 Front-end............................................................................................................................................ 12

3. SCENARIOS/STORIES .................................................................................................................14

3.1 SP SCENARIO 2: FILLING OF AN ACTIVATED FEED (EVENT DRIVEN) ...........................................14

3.2 HA SCENARIO 1: SINGLE USER 2-SCREEN VIEW OF THE ENHANCED PERSONALIZED NEWSCAST

(USER DRIVEN) ..................................................................................................................................17

3.3 HA SCENARIO 2: TWO USER 2-SCREEN VIEW OF THE ENHANCED PERSONALIZED NEWSCAST

(USER DRIVEN) ..................................................................................................................................22

3.4 HA SCENARIO 3: USER SEARCH (USER DRIVEN) ..........................................................................23

4. GUIDED WALKTHROUGH ........................................................................................................25

4.1 SERVICE PROVIDER USER INTERFACE .........................................................................................25

4.2 HOME AMBIENT USER INTERFACE ..............................................................................................26

5. IMPLEMENTATION .....................................................................................................................35

5.1 SERVICE PROVIDER SIDE .............................................................................................................35 5.2 HOME AMBIENT SIDE ..................................................................................................................35

6. CURRENT STATUS AND FUTURE WORK..............................................................................36

REFERENCES ....................................................................................................................................36

FP7 - 231761

Deliverable 7a.4

of 36

LIST OF FIGURES Fig. 1 - NoTube WP7a Service Provider side architecture .................................................. 11

Fig. 2 - NoTube WP7a Home Ambient side architecture ..................................................... 12

Fig. 3 -NoTube WP7a Home Ambient Front-end ................................................................ 13

Fig. 4 - Swimlane: SP scenario 2 - Acquisition phase ......................................................... 15

Fig. 5 - Swimlane: SP scenario 2 (NIC Management phase) .............................................. 16

Fig. 6 - Swimlane: SP scenario 2 (Feed filling phase) ......................................................... 17

Fig. 7 - Swimlane: HA scenario 1 (Login and personalised list page) .................................. 18

Fig. 8 - Swimlane: HA Scenario 1 (Info page access) ......................................................... 19

Fig. 9 - Swimlane: HA Scenario 1 (Play HQ video on TV set) ............................................. 19

Fig. 10 - Swimlane: HA Scenario 1 (TV set remote control) ................................................ 20

Fig. 11- Swimlane: HA Scenario 1 (External references) .................................................... 20

Fig. 12 - Swimlane: HA Scenario 1 (Suggestions list) ......................................................... 21

Fig. 13 - Swimlane: Scenario 1 (Send suggestion).............................................................. 21

Fig. 14 - Swimlane: HA Scenario 2 (Local sharing) ............................................................. 22

Fig. 15 - Swimlane: HA Scenario 3 (My news search) ........................................................ 23

Fig. 16 - Swimlane: HA Scenario 3 (Home search) ............................................................. 24

Fig. 17 – Management of new or old feeds ......................................................................... 25

Fig. 18 – New metadata for the feed ................................................................................... 25

Fig. 19 – Smartphone samples ........................................................................................... 26

Fig. 20 – Integrated login using the QR Code .................................................................... 27

Fig. 21 - Welcome message ............................................................................................... 28

Fig. 22 - The personalised list of news ................................................................................ 28

Fig. 23 - First page of a particular News Item ...................................................................... 29

Fig. 24 – Related concept list .............................................................................................. 29

Fig. 25 - The browsing for external related links .................................................................. 30

Fig. 26 – The main video on the TV screen ........................................................................ 30

Fig. 27 - Sharing news items with friends (sending) ............................................................ 31

Fig. 28 - Sharing news items with friends – the message in Facebook ............................... 31

Fig. 29 - Receiving suggested news items .......................................................................... 32

Fig. 30 - The search input form ........................................................................................... 32

Fig. 31 - Sharing news items between two or more user watching the same TV set ........... 33

Fig. 32 - News Item details with play out feature on mobile device ..................................... 33

Fig. 33 - Reframed video play out on mobile device ............................................................ 34

FP7 - 231761

Deliverable 7a.4

of 36

LIST OF TABLES

FP7 - 231761

Deliverable 7a.4

of 36

List of Acronyms and abbreviations

Acronym Description

A/V Audio Video ANTS Automatic News Transcription System API Application Programming Interface CRID Content Reference Identifier DB Data Base EBU European Broadcasting Union eNIC Enriched News Item Container EPG Electronic Program Guide FOAF Friend Of A Friend project GUI Graphical User Interface HA Home Ambient HMI Human Machine Interface HTTPS Hypertext Transfer Protocol over Secure Socket Layer leNIC Locally Enriched News Item Container MSQC Main Stream Quality Content NIC News Item Container OAID On Air (program) IDentifier PC Personal Computer PDA Personal Digital Assistant PVR Personal Video Recorder PWD Password RDF Resource Description Framework REST Representational Transfer State RSS Really Simple Syndication SOA Service-oriented Architecture SP Service Provider SPARQL SPARQL Protocol and RDF Query Language SQL Structured Query Language STB Set-Top Box UML Unified Modeling Language URI Universal Resource Identifier URL Uniform Resource Locator WMV Windows Media Video

FP7 - 231761

Deliverable 7a.4

of 36

1. Introduction

1.1 Scope of this document This document is a paper companion to the software prototype D7a.4 Personalized News Platform prototype v3. In this deliverable all issues are faced from a technical point of view rather than from a logical point of view, leaving the last one to the deliverable [1].

1.2 Document outline In section 2. we introduce an overview of the integrated architecture, focusing on the two main sides, the Service Provider side and the Home Ambient side. For reference purposes we also highlight the integration with services developed in other NoTube Work Packages, also making reference to the NoTube Integrated Platform described in [2] and [3].

In section 3. we take into account some reference scenarios making reference to the actual use cases that have been implemented in this third release of the prototype.

Section 4. is a guided walkthrough of the prototype where some step by step GUI screenshots are used to better describe the whole prototype operations.

In Section 5. Main implementation upgrades about this 3rd release of the prototype are presented.

In Section 6. an overview of current status and possible future works is presented.

FP7 - 231761

Deliverable 7a.4

of 36

2. Prototype Architecture The third prototype (M33) implementation of the Personalized News Platform Use Case adheres to the high-level architecture design as sketched in [2] and [3], both for the SP (Service Provider) side and the HA (Home Ambient) side.

2.1 Service Provider As to Service Provider side we can highlight two different environments. The Legacy one refers to systems and components which are used by the NoTube platform and which are already existing independently from the platform itself (proprietary systems). The NoTube one refers to systems and components which directly belong to the NoTube platform: in more details there is a set of components and systems which implements the particular actual service using a second set of general services provided by other workpackages under the broker mediation. With respect to the second prototype [4], the cropping, metadata transformation and NIC CRUD services have been fully integrated.

Fig. 1 - NoTube WP7a Service Provider side architecture

2.2 Home Ambient

2.2.1 Back-end

The Home Ambient component (HA), on the other hand, is connected with:

1. The Service Provider, through the internal HA/SP APIs basing on an hybrid notification mechanism that leverages on RSS feeds and HTTPS-based data exchanges.

2. The Front-End, through another set of APIs internally exposed through SOAP-based Web Services for what concerning user profile management and News recommendation services. Media files (News) are accessed directly via the HA persistence layer.

NIC RepositoryCONTENT

Repository

RULES Repository

RULES MANAGER NIC MANAGER SERVICE MANAGER

SEMANTIC BROKER

WP5

METADATA

TRANSFORMATION

SERVICES

WP2

NIC CRUD

SERVICES

WP2

CONTENT

MANAGEMENT

SERVICE

EPG SERVICES

WP1

LUPEDIA

SERVICE

WP4

FEED BUFFER

LOCAL CRID

RESOLUTION

SERVICE

NIC REPOSITORY

SERVICE

ACQUISITION

BUFFERS

ACQUISITION SYSTEM

I/F

Legacy Environment

ANTS

CROPPING

SERVICE

WP4

NoTube Environment

BROADCAST FEED

SPArchitecture-3.00

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 2 - NoTube WP7a Home Ambient side architecture

Referring to the diagram above, the internal workflow implements the ingestion of Newscasts (or individual News, if required) from both the IP channel and the standard broadcasting, through the component named “LiveIngestion”.

The “NewsOrganiser Agent”, then, is aimed at managing, extracting and resorting the ingested audiovisual contents together with the acquired metadata, when made available by the Service Provider, in order to locally store a set of single News for both the audiovisual and metadata component. Moreover, at this stage, the user profile and recommendation services provided by WP3 are integrated running in background threads responsible of dynamically building up the playlists for the personalized Newscast of each active user. In this third prototype a Lucene indexer has been integrated in order to generate "Home" Lucene persistent indexes useful to search purposes: every time a new news item comes in the ambient these indexes are updated. These indexes are stored inside the "News Items Metadata Repository".

The News Item Container (NIC) persistence and content resolution layer has been implemented similarly to the Service Provider, moreover a number of internal refinements have been developed to reflect Service Providers’ improvements.

The HA back-end, thanks to the mentioned integration of WP3 user profile manager (including the connection with the Beancounter) and the News recommendation services built on top of the user activities logged from social networks, is capable of delivering a personalized playlist to the front-end module.

The front-end has been redesigned completely, developed and integrated, enabling the end-user to perform standard tasks such as logging on and modifying his/her profile and interests as well as consuming the personalized Newscasts surrounded by enriched metadata enclosed in the NIC.

2.2.2 Front-end

The Home Ambient front-end can be split in two different parts: the first one belonging to the Home Ambient Server and the second one related to the different user devices, as shown in the following figure.

News Items

Metadata

Repository

News Items

A/V

Repository

HA/SP

APINews Organiser Agent Front-End API

METADATA

TRANSORMATION

WP2

EPG

WP1

LOCAL CRID

RESOLUTION

SERVICE

Content

Ingestion

Buffer

Live Ingestion

Service Provider

Home Ambient

NoTube Services

FRONT-END

SERVICE

MANAGER

Broadcast Feed

Metadata

Ingestion

Buffer

USER PROFILE

MANAGEMENT

WP3

NEWS

RECOMMENDATION

WP3

Asset

Persistence API

Am

Ife

ed

ba

ck

Lucene

Indexer

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 3 -NoTube WP7a Home Ambient Front-end

The Home Ambient server front-end uses the Front-end API to exchange data with the back-end for what concerning user profile management and News recommendation services. Media and metadata files on News Items Repositories are accessed directly via the HA persistence layer.

The Web Server modules is the core of the front-end environment: it receives request from the handheld device (second screen) and interacts with other software modules in order to send dynamic web pages to the second screen and to control the video playing on the TV set (primary screen).

The Remote Control Unit module is in charge of send the correct play commands to TV set module which in turns convert them in suitable commands for the TV set (in this prototype a custom protocol over UDP is used for communications between the two modules).

The Facebook dispatcher is the interface between the WEB server module and Facebook, it allows the system to publish suggestion as posts on user wall and, in reading mode, it allows retrieving the NoTube posts published in the facebook home page, using Facebook APIs.

The HQ content delivery module exposes the high quality contents (in the third prototype all tasks of this module are covered by the Microsoft shared directory software).

In order to implement the search service, two search modules based on Lucene API have been developed: the first one, Lucene indexer, creates Lucene "in RAM" indexes related only to the user personal news items (already recommended using WP3 features), while the second one, Lucene searcher, executes searches using these "in RAM" indexes or using more extended "home" indexes (created in the home ambient sever back-end) if the user wants to search the whole home ambient content. At the moment, news item indexes are built over the Speech to Text fields.

Home Ambient Server

Front-end

User Composed Device

Back-end

TV set

WEB Server

module

Remote

Control Unit

module

TV Set

moduleTV Screen

News Items

A/V

Repository

HQ content

delivery

module

Front-End

API

Handheld device

WEB

Browser

Touch

Display

Custom protocol

over UDP

http

CIFS

FEArchitecture-3.02

Lucene

Indexer

Lucene

Searcher

Facebook

Dispatcher

Facebook

Graph API

Related WEB

SitesDBpedia site

QR-Code

reader

News Items

metadata

Repository

FP7 - 231761

Deliverable 7a.4

of 36

3. Scenarios/Stories

3.1 SP scenario 2: filling of an activated feed (Event driven) This scenario deals with operations the SP system does in order to fill an actual feed to expose to HA systems. “The SP system, once a feed is activated, monitors the broadcast channel waiting for the time to start recording each newscast belonging to the feed itself (Acquisition phase). At every expected time the different systems processes used to ingest and create (e.g. cropped version of the ingested material) contents and metadata start and at the end of these processes the SP system is able to automatically create NICs to store into the NoTube platform. Afterwards each NIC is automatically enriched by the system with metadata and contents related to the news described into the NIC itself (NIC Management phase). Following rules relevant to the examined feed, which specify the genres of interest for it, the SP system selects among all NICs only those which are suitable for the feed itself. In this way, each time the SP system can fill the actual feed with the list of selected NICs identifiers, update and expose it to HAs (Feed filling phase). On the other side a HA feed reader monitors the feed and is able to access to it every time it is updated with new NICs identifiers.” The three phases of the scenario are described by the following three swimlanes diagrams: Acquisition phase in Fig. 4, NIC Management phase in Fig. 5 and Feed filling phase in Fig. 6. In particular, the updates refer to the introduction of the TVN cropping service in the acquisition phase and the whole integration of the WP2 NIC CRUD and TV-A transformation services in the NIC management phase. For additional information about the video reframing functionality see [5].

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 4 - Swimlane: SP scenario 2 - Acquisition phase

Broker

Acquire program in HQ

formats

Selected program is

on air

Broadcast feedAcquisition

systemANTS

Acquire program in LQ

formats

Store HQ contents

Save LQ contents

Store LQ contents

Detect News Items

Create program description file in

Presto Space format

Save program description file

Store program description file

Wait for a new selected

program

Wait for a new selected

program

Scenario2(1)-2.00

Save HQ contents

Acquisition

Buffers

NoTube

Services

CROPPING Enrichment

RequestCROPPING Enrichment

TVN Cropping Service

Send Cropped content

Save Cropped content

Store Cropped contents

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 5 - Swimlane: SP scenario 2 (NIC Management phase)

A new program description file is

present

Request program

description file

NIC Manager NIC Repository

PrestoSpace to TVA format

transformation

Request transformation for ANTS file

Send set of NICs in NoTube

format

NIC CRUD service

Request NIC internal data modification

Send modified NIC

Store ingested NIC contents

Store NIC

Create basic NICs

LUPEDIA Enrichment

NIC CRUD service

Request NIC internal data

access

Send NIC internal data

LUPEDIA enrichment

service

Request LUPEDIA

enrichment

Send enrichment

Send program description file

(ANTS file)

BrokerNoTube

services

Customize basic NIC,make Basic

Content Enrichment

and save NIC

Send ingested NIC contents

Request NIC Send NIC

NIC CRUD service

Request NIC internal data modification

Send modified NIC

Store NIC

Wait for a new program

description file

Scenario2(2)-3.00

Acquisition

buffers

Content

Repository

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 6 - Swimlane: SP scenario 2 (Feed filling phase)

3.2 HA scenario 1: Single user 2-screen view of the Enhanced Personalized newscast (User driven) “Roberto is a family man, working honorably for his company since almost 30 years. As every evening, he's now back from work and relaxing on his sofa watching to his personalized newscast provided by the NoTube platform on his new smartphone. After logging himself on the home ambient and on the TV set in front of his sofa with the QR Code system, he takes a look at the list of news that the system has selected for him, moving through different genres of news presented on his smartphone when his attention is captured by a news into his personalised list broadcasted on the TG1 of 13 a.m. talking about new modalities for the retirement which will be applied from next year on (Fig. 7). He decides to analyse this news carefully to know how rules are going to change...he goes to the news general info page (Fig. 8) and starts watching the cropped video of this news on his smartphone, but it could be better to see it bigger and with a simple click he sends the main quality video on the TV set (Fig. 9) and controls it using his smartphone as a remote control (Fig. 10). While watching he realizes the speaker is talking about something he doesn't know exactly, so leaving the video playing on the TV set he can click on that concept into the related concepts inner page on the smartphone in order to see an additional explanation about the concept itself (Fig. 11). This news is really interesting for his future life, so it could be useful to check from the list of external links if there is any related news item coming from the web, he chooses some from the related external links inner page on the smartphone and reads them greedily while stopping the video on the TV to pay more attention on what he's reading (Fig. 11). After finishing the play of the video, he searches into the list of news suggested by friends on Facebook if there are other news about the same theme (Fig. 12) ... here's one, coming from the

OK

A new enriched NIC is present

Request RSS feed

(Meta feed)

Send current feed xml file

Insert NIC id into feed xml

file

Publish RSS feed

(Meta feed)

Store modified xml file

Wait for a new

enriched NIC

Service

ManagerNIC Repository Feed Buffer

Scenario2(3)-2.00

Rules

Repository

Request feed related

program to acquire list

Send program to acquire list

Verify NIC suitability versus

“input rules”

Verify NIC suitability versus “service rules”

OK

FP7 - 231761

Deliverable 7a.4

of 36

evening edition of TG1, Roberto doesn't miss the chance to see also this news to verify if there are other info to know about his future! If possible, this last news is more interesting than the previous one, Roberto decides to share it with friends using the Facebook sharing service provided by the NoTube platform (Fig. 13). Now he really knows everything about this topic, maybe shortly he will get a pension too! More relaxed he can keep on looking for other interesting news into his personalised list”. The whole scenario is described by the following swimlanes diagrams.

Fig. 7 - Swimlane: HA scenario 1 (Login and personalised list page)

TV Screen QR-Code reader WEB BrowserWEB Server

moduleFront-End API

News Items

Repositories

Show QR-Code

Acquire QR-Code

Start browser on QR-Coded

URL

Request QR-Coded URL

page

Send user/device

info

Request user suitable news

items list

Send user suitable news

items list

Build Personalized

“by genre” list

Build response

page

Process List page

Provide contents and

metadata

Touch Display

Show Personalized

List page

D7a.4- HA1(a)-1.00

Click on QR-Code Reader

Select inner list page

Process List page

Show selected list

page

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 8 - Swimlane: HA Scenario 1 (Info page access)

Fig. 9 - Swimlane: HA Scenario 1 (Play HQ video on TV set)

WEB BrowserWEB Server

module

News Items

Repositories

Click on aList element

Request Info page


metadata

Touch Display

Show Info page

D7a.4- HA1(i)-1.00

Build Info page

Process Info page

Select inner page

Process Info page

Show selected

inner page


module

HQ content

delivery module

News Items

Repositories

Click onPlayOnTVButton

Request Remote

Control page

Request play action on

selected item

Send LOAD&PLAY

command

Touch Display

Show Remote

Control page

D7a.4- HA1(c)-1.00

Remote Control

Unit moduleTV Screen TV Set module

Receive LOAD&PLAY

command

Access HQ content

Execute PLAY

Provide HQ content

Show HQ video

Build Remote Control Page

Process Remote

Control Page

Reach HQ content

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 10 - Swimlane: HA Scenario 1 (TV set remote control)

Fig. 11- Swimlane: HA Scenario 1 (External references)


module

Click on“command “ on Remote Control

page

Request Remote

Control page

Request “command”

execution

Send “command”

Touch Display

Show Remote

Control page

D7a.4- HA1(d)-1.00

Remote Control

Unit moduleTV Screen TV Set module

Receive “command”

Execute “command” on playing

video

Show HQ video

Build Remote Control Page

Process Remote

Control Page

WEB Browser

External WEB

server or

Dbpedia server

Click onExternal

link/concept

Request External page

Touch Display

D7a.4- HA1(h)-1.00

Send response

page

Process External page

ShowExternal

page

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 12 - Swimlane: HA Scenario 1 (Suggestions list)

Fig. 13 - Swimlane: Scenario 1 (Send suggestion)


module

Facebook

Graph API

News Items

Repositories

Click onSuggestions List button

Request Suggestions

List page

Request Facebook

suggestions

Request user stream on Facebook

Read &Send user stream

Build suggestions

list

Build response

page

Process Suggestions

List page


metadata

Touch Display

Show Suggestions

List page

D7a.4- HA1(b)-1.00

Facebook

Dispatcher

Select suggestions from stream


module

Facebook

Graph API

News Items

Repositories

Request suggestionpublication

Request publication on user wall

Publish on user wall

Build response

page

Process response

page


metadata

Touch Display

Show response

page

D7a.4- HA1(l)-1.00

Facebook

Dispatcher

Get publication response

Click onSuggest button

Request Suggest

page

Build Suggest

page

Process Suggest

page

Show Suggest

page

Write comment

Select post privacy

FP7 - 231761

Deliverable 7a.4

of 36

3.3 HA scenario 2: Two user 2-screen view of the Enhanced Personalized newscast (User driven) “Sitting next to Roberto on the sofa is his wife, Maria; she is also focused on her personalised newscast provided by the NoTube platform on her smartphone. She is near to the retirement too, and would be very interested to see the same news her husband is watching at on his smartphone that unfortunately is not into her list of recommended news; Roberto is looking at the cropped video on his smartphone at the moment..."Could you please send the video on the TV?" - asks Maria to her husband - "so that I can get that news on my own smartphone too?". Once the video is playing on the TV, since she's also logged on that same TV set, she can get the general info page of the news on her own smartphone and enjoy this news with all the enrichments exactly as her husband (Fig. 14) ...and unfortunately for him she can also use her smartphone as remote control for that video...”

Fig. 14 - Swimlane: HA Scenario 2 (Local sharing)

TV Screen WEB BrowserWEB Server

module

News Items

Repositories

Show HQ video

Touch Display

D7a.4- HA1(L)-1.00

Click on Info button

Request Info page


metadata

Show Info page

Build Info page

Process Info page

Select inner page

Process Info page

Show selected

inner page

FP7 - 231761

Deliverable 7a.4

of 36

3.4 HA scenario 3: User search (user driven) “Roberto is a busy man at home too, sometimes he has not a lot of time to see all the personalised news, so he prefers to explicitly ask to the system to give to him a list of news about particular topics...for this purpose he likes using the search service provided by the NoTube platform, he can ask for NICs talking about specific topics into his own personalised list of news (Fig. 15) or into the whole home ambient (Fig. 16), in any case he can always find what he's searching for!”

Fig. 15 - Swimlane: HA Scenario 3 (My news search)


module

News Items

Repositories

Click onSearch button

Request Search page

Request Search

operation

Search using “in RAM”

Index

Build Search Results List

Build response

page

Process Suggestions

List page


metadata

Touch Display

Show Suggestions

List pageD7a.4- HA1(f)-1.00

Lucene

Searcher

Build response

page

Process Search page

Show Search page

Write search string

Request Search

Results page

Select “My news” mode

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 16 - Swimlane: HA Scenario 3 (Home search)


module

News Items

Repositories

Click onSearch button

Request Search page

Request Search

operation

Search using “Home”

Index

Build Search Results List

Build response

page

Process Suggestions

List page


metadata

Touch Display

Show Suggestions

List pageD7a.4- HA1(g)-1.00

Lucene

Searcher

Build response

page

Process Search page

Show Search page

Write search string

Request Search

Results page

Select “Home news”

mode

FP7 - 231761

Deliverable 7a.4

of 36

4. Guided walkthrough

4.1 Service Provider User interface The SP front end application presented for the third prototype is the enhanced version of the one developed for the second prototype. Please refer to [4] to read about Year 2 functionalities. Here are the enhancement provided in Year 3 at Service Provider side.

Fig. 17 – Management of new or old feeds

The user can choose if creating a new feed with new SP rules or if managing an already existing feed to change only some particular characteristic of the feed itself (Fig. 17).

Fig. 18 shows two new metadata about the feed: the duration of the feed, which is used to know how long the feed will be operative for and which drives the recording of newscast by the system, so that when the feed stops to be operative, the broadcasted newscast foreseen for that feed are no more automatically recorded, and the window of contents validity, which is used to know after how long contents are considered “old” and could be deleted by the system.

Fig. 18 – New metadata for the feed

FP7 - 231761

Deliverable 7a.4

of 36

4.2 Home Ambient User Interface

The front-end application is designed to run on a smartphone (as those presented in Fig. 19) used as secondary screen, in particular this third prototype is showed on the smartphone of Fig. 24.

Fig. 19 – Smartphone samples

In the third prototype the HA and the device (TV set) identification are exposed through a QR Code. The user can access to the service with his smartphone using the authentication integrated with the ambient intelligence inside the Home Ambient.

FP7 - 231761

Deliverable 7a.4

of 36

The user, handling his smartphone, catches the HA and the TV set ID and is automatically connected to the ambient and automatically identified by it using a device/user fixed association (Fig. 20).

Fig. 20 – Integrated login using the QR Code

When the user enters into the NoTube ambient a welcome message is displayed for few seconds (Fig. 21) and the user receives his personalised news list that has been automatically edited following his own preferences; these can be dynamically extracted following the behaviour of the user in the social network or can be fixed by the user himself through the selection of its favourite genres. The personalised list of news is presented organised by genre, each genre is achievable by scrolling to the left and to the right. Each genre list is top-down scrollable and a genre banner is always visible in the screen as showed in Fig. 22.

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 21 - Welcome message

Fig. 22 - The personalised list of news

By tapping on a particular news item the first page containing general information is presented as shown in Fig. 23.

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 23 - First page of a particular News Item

Scrolling horizontally the prototype allows the viewing of a second information inner page where concepts that are considered significant inside the news, extracted by the Lupedia service[6], are presented as a list; clicking on one item (see Fig. 24) the relevant DBpedia page will be displayed on the screen.

Fig. 24 – Related concept list

The NIC structure allows the presence of external related links that can be presented into the prototype in a third inner information page, where a set of URL related to a particular news item is

FP7 - 231761

Deliverable 7a.4

of 36

available for the navigation in web sites of other media companies that expose information about the same news. The user can navigate through the related external links and touching an item from the list the browser reaches the relevant URL showing the related news (Fig. 25).

Fig. 25 - The browsing for external related links

The third prototype allows the final user to see the main video on the TV screen. Into each of the three information pages there is a button used to play the news related main video on the TV set. At the same time on the smartphone a remote control is presented so that the user can remotely control the play (Fig. 26).

Fig. 26 – The main video on the TV screen

Inside the prototype the user is able to share a particular News Item with friends in Facebook (Fig. 27 - Fig. 28 - Fig. 29).

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 27 - Sharing news items with friends (sending)

Fig. 28 - Sharing news items with friends – the message in Facebook

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 29 - Receiving suggested news items

Inside the third prototype the search feature is added to allow the user to perform searching of news inside the personalised list or inside all news available in the Home ambient (Fig. 30).

Fig. 30 - The search input form

FP7 - 231761

Deliverable 7a.4

of 36

The application allows different users authenticated on the same HA and TV screen to get the played NIC. Using this feature it is possible to share news items between two or more user watching the same TV set. Fig. 31 shows a different user with a different device that is able to acquire the control and all news information of a news item played on a TV set by a different user.

Fig. 31 - Sharing news items between two or more user watching the same TV set

The video reframing functionality based on ROI technology is integrated in the Personalized Semantic News use case.

In particular, the reframed video of News items is used for the play out of a particular news on the smartphone display (Fig. 32 - Fig. 33). When the final user selects a particular news he can choose if playing it on the TV screen or to watch it directly on the smartphone screen.

For additional information about the video reframing functionality see [5].

Fig. 32 - News Item details with play out feature on mobile device

FP7 - 231761

Deliverable 7a.4

of 36

Fig. 33 - Reframed video play out on mobile device

FP7 - 231761

Deliverable 7a.4

of 36

5. Implementation

5.1 Service Provider side The main improvements brought in the 3rd prototype at service provider side are:

• Feed management o Period of working of a feed o Validity period for contents of a feed

• MSQC live ingestion o Automatization of the whole process o Automatic creation of transport stream files for cropping o Automatic creation of cropped files

• Integration of metadata transformation services • Integration of NIC CRUD services

The SP feed management application is developed to run on a PC. In particular, the SP GUI is a windows application developed with MS Visual Studio 2008/2010.

5.2 Home Ambient side Functionalities developed in the 3rd prototype are:

• QR Code integration for HA registration • Authentication using cookies inside the telephone • News Item Container video player for smartphones and for TV set • Facebook sharing of multimedia objects for sharing of News items with friends from the

smartphone • Facebook friend suggestion in the smartphone • Search functionality (HA and personalized) • Sharing of News Item in the same HA • Showing DBpedia concepts on smartphone • Management of external related links on smartphone

The developing tools used to build the prototype is MS Visual Studio 2010 integrated with Facebook.sdk, Lucene and SimpleLucene.

FP7 - 231761

Deliverable 7a.4

of 36

6. Current status and future work For the Service provide side we have finished the integration of the whole NoTube platform.

About the Application for the final user, the prototype described in this document covers all requirements for a television application in the environment of the Personalized News Service using secondary screen features. The future work on next three months foresees some minor improvements on both the Service Provider side and the user application.

The future work after the end of the project could be highly related to the extension of the Model/Architecture developed in the NoTube project to all types of television content and not only based on news items. In this direction the already developed software could be reused and improved in order to make the prototype a real service for the final user.

References

[1] L. Vignaroli, F. Negro, R. Del Pero et al., D7a.1 Personalized News Services Specifications, 2009

[2] A. Conconi, F. Cattaneo et al., D6.1b NoTube System Specifications and Architectural Design, 2009

[3] A. Conconi, F. Cattaneo et al., D6.3 NoTube Integrated System 2nd Prototype , 2010

[4] L. Vignaroli, F. Negro, R. Del Pero et al., D7a.3 Personalized News Platform prototype, v.3, 2010

[5] A. Mevel, L. Vignaroli, F. Negro, R. Del Pero, G. Spikofski, J. Groh, M. Elser, P. Altendorf, D4.3v3 Audio/Video content analysis component, 2011

[6] P. Mihaylov, D4.5 Integration of Advanced Modules in the Annotation Framework, 2011

D7a.4 Personalized News Platform prototype v3 · FP7 - 231761 Deliverable 7a.4 Page 1 of 36 NoTube...

Documents

Transcript of D7a.4 Personalized News Platform prototype v3 · FP7 - 231761 Deliverable 7a.4 Page 1 of 36 NoTube...