1

Prof.dr. Inald Lagendijk, Coordinator

Delft University of TechnologyFaculty of Electrical Engineering, Mathematics and Computer Science

Network of Excellence PetaMedia

Peer-to-Peer Tagged Media

(42 months, started March 1, 2008)

http://www.petamedia.eu

2

Partner Concept of PetaMedia: EU Network of National Networks

TUD QMUL

EPFL TUB

Interested partners

(IPG)EU Virtual Center of Excellence

PermanentAffiliated partners

(NIRICT)

PermanentAffiliated partners

(MMKM)

PermanentAffiliated partners

(IM2)

PermanentAffiliated partners

(HC3)

3

Technical Scope of PetaMedia

• What: Developing new paradigms and technologies for efficient

and effective access to multimedia contentHow: Multimedia content analysis meets P2P/social networks

Best of three worlds

Peer to peer/social networks(Tribler software)

User-based taggingContent analysis

4

Objectives of NoE PetaMedia

Four main objectives:

1. European network formation and strengthening in the area of multimedia content access. Consolidation of the network beyond lifetime of NoE as Virtual Centre of Excellence.

2. Progress state of art in (the combination of) user tagging - content analysis - P2P/social networking via joint research.

3. Converge of results to demonstration and experimentation.

4. Dissemination of knowledge (P2P technology, content analysis, information retrieval, …) inside and outside NoE.

5

Example of Scenario and Research (1)

6

Example of Scenario and Research (2)

user 1: tree sun dog cloud

user 2: tree sun vacation cloud dog clouds

score: tree sun dog cloud

7

Example of Scenario and Research (3)

• Temporal tag integration (learning)• Temporal generalization (tag a frame given tagged

neighboring frames)• Temporal segmentation (into tag-based shots)

8

Example of Scenario and Research (4)

• Object detection: Spatially connect tags to objects• Ask user to collaboratively click on objects, given the

temporal tags

• Use resulting spatio-temporal tags demonstration/field tests • Video segment (shot) retrieval• Browsing• Recommendation

9

Build on Tribler P2P Software (1)• Value creation and intelligence goes to the edges: P2P• Tribler: Operational P2P system, started Feb 2006• Long term, experimental living lab

• Largest dataset collection

• Empirical approach

• 170.000+ Tribler downloads

• Open Source & community building

• Social File sharing

• Unified search & viewing

• Bittorrent, Youtube.com, Liveleak.com

• Distributed sharing, rating, recommendation

• Research team: 18 full-time people

10

11

Build on Tribler P2P Software (2)

• Tribler core toolkit and API

• Tribler workshop @ Delft: May 27, 2008 • More: http://www.tribler.org

IP Layer

Tribler core toolkit and APIP2P peer discovery recommendation download/stream

Appl. Specific #1 Appl. Specific #2 Appl. Specific #3

12

Special Interest Groups Formed during KickOff• Content/tag (nonlinear) life cycle and required supporting (distributed)

architecture, incl. cooperative content authoring• Need for distributed processing and content swapping (in particular

scenarios)• Automatic object and/or action segmentation and recognition

• Keep-it-simple user interface supporting tagging and tag-use• Incentives to tag: games, type of content, social tagging (friends),

contesting and reputation

• Experience tagging – beyond visual cues: sensor-based tagging (camera, microphone, bio, …)

• Group intelligence building: trust, reputation, friends, cooperative authority

• Correctness/quality/originator/evaluation of tags and ways of feedback (create trust)

• Prepare tags, use tags, link tags: recommendation, personalization, profiling, indexing of video segments

13

Special Interest Groups Formed during KickOff

14

May 27

May 27