Thinking the archives of 2020: Opportunities, priorities, issues

An exchange between FIAT/IFTA Members for Benchmark, Synergies and to promote Standards

Gerhard Stanz (ORF), Yasuhiko Iwasaki (NHK), Alberto Messina (RAI), Theo Mäusli (SRG)

ScheduleFriday 9 october

16.00-16.10 Introduction to the main challenges: questions, state of the art and opportunities on technical, organisation and political issues

Theo Mäusli

During 16.10-17.15 2020: What are your main Visions, Priorities, Roadmap, Worst scenario: send your list priorities and new suggestions, via Directpoll and twitter.

All, twitting and sending answers to Directpoll

16.10-16.20 NHK, priorities and main issue Yasuhiko Iwasaki

16.20-16.30 RAI, as above Alberto Messina

16.30-16.40 ORF, as above Gerhard Stanz16.40-16.45 SRG SSR, as above Theo Mäusli16.45-17.30 Questions and common points, discussion, developing together a

topic list with priorities, possible roadmaps and main approachesall

The Broadcaster Archivist’s Cahier de Doléances

for Voting: max 5 x/votehttp://etc.ch/z2Ic

Storage

Formats

Rightsmanagement

Traceability

Metadataautomation

User interface

Workflows

Storage medium and

migration

Architecture

Rightsmanagement

FinancingOwnershipt

Enhancement

Storage and migration

• Magnetic or optical, … DNA?

• Backup solution?

• Cloud?

• …

Storage medium and

migration

https://www.tbunews.com/wp-content/uploads/2012/09/dna-storage-451x292.jpg

Architecture

• One MAM System

• Different modules, Apps, SaaS, linked with a Bus

• External services?

Architecture?

Formats

• Store and deliver only standard format

• Store one format, delivering passing througha transcoding service

• Store and deliver multiple formats

• …

FormatsFor convergent use

Rights management

• Automation of workflows from SIP to DIP

• Promote free use of archives?

• National laws and agreements

• Creative commons?

Rightsmanagement

Rightsmanagement

User interface

• New generations of searching tools

• Individual user profiles?

• Push – pull?

• Desk services?

• More exchange between archives?

User interface

Enhancement

• A new activity of archivists?

• Main criteria for success and

• Sustainability

Enhancement

Metadata automation

• Speech 2 text• Image and face recognition• Ontologie• GPS• Production metadata workflow• Social tagging• Cross all the Technologies and opportunities

Metadataautomation

Traceability

• Fingerprint

• Mpeg 7

• …

Traceability

http://www.bing.com/images/search?q=fingerprint&view=detailv2&&id=E59C7D50D97A11A38DF883D3EA867D0FB42312BA&selectedIndex=43&ccid=pY4d6xKk&simid=608014941461811131&thid=OIP.Ma58e1deb12a49974474fe0f6ccb50ce2o0&ajaxhist=0

Financing

Financed

• by programme and externals, when using archive material

• by producers for archiving their content

• as overhead of the whole institution

• by cultural heritage founds (law)

Financing

Ownership

• National audio-visual archives

• Another facet of Service public

• A private asset

• Mixed approaches (network between Broadcasters and cultural heritage institutions)

OwnershipBroadcaster or

government https://upload.wikimedia.org/wikipedia/commons/1/19/Franzosen_Staatsschatz.jpg

Thinking the archives of 2020: Opportunities, priorities, issues

Yasuhiko Iwasaki (NHK)

Possible Change of…

• Technology– Big Data Analyzing– Automatic metadata– Adaption to Multi Device Market– Expanding of archiving material

• Social Network– How to get “Like” on SNS? Or How to be conspicuous ?– User Generated Contents– Context Mining for rich navigation

• Organizational and political issues– Copy Rights, etc

Bipolarization of Future Archives

High Resolution

8K(4K)HDR CinemaHigh Res Movie

Light Resolution&

Rich NavigationUbiquitousMulti PlatformUser GeneratedOn Demand

Two major cost pressure is…1. Meta data management2. Storage and BandwidthOptimized Preservation Transcodable Preservation

It's NOT just the tip of the iceberg

Value for market

CostSank to oblivion

How to balance the preservation cost and it’s value….That is the question!!

How to decide our Priority ?

1. Responding to further increase of preservation quantity

2. Responding to increasing transfer rate for the ultra high-resolution material

1. Metadata and tagging work for the pre-digital age content

2. Responding to a multi-platform market.

3. Responding to changes in the role of broadcasters.

Value centric strategy requires:

Preserve centric strategy requires:

It Depends…• Each archive organization has its strategy of

“choice and focus”.

• NHK is more focusing to “Public Media” from “Public Broadcaster” for now.– That means more optimizing to services for Over

IP type…

• But look for 10 to 20 years, super high resolution is also important.

• This Challenge is mutually contradictory.

Thinking the archives of 2020

Gerhard Stanz

Archives 2020

Archive Object

• Programmes / Products• There is still a lot of „old“ in the „new“• But still we expect …

• more Versioning / Granularity• more Crossmedia• more Metadata involved

• EPG, Recommender-Systems, Second Screens• Archiving Websites prototype for New Challenges

• Presentation Layer vs. Raw-Material• An Archive is not necessarily an Encyclopedia (which you can

anyhow find in the Internet)

Archives 2020

Metadata Automation

• Harvesting• Data that are alraedy there in the production process

• Higly accurate and appropriate• Teleprompter Text, Insert-Text, Texts froom Planning

Systems, Subtitles• Mining

• Texts derived from other „Media representations“• False Positives, False Negatives• Relyability vs. Cost

• Adobe hast discontinued „Speech to Text in“ 2014• Manual Annotation

• „Humans as a Service“• Main cataloguing effort in ORF is „image description“ of

Material with extensive ORF-rights

Archives 2020

Storage Security

• Aspects of Security• Operational Security• Content Security• Disaster Security

• Instances / Redundancies• 1. Online• 2. Online (Operational, Content)• 3. Offline (Content / Disaster)

• Moste likely Disasster Szenarios• 2nd Disaster during Rebuild of 1st Redundancy• Unrecognized Software Errors

©

Thank you for your attention

RAI

• Please Alberto, insert your slides

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

29

RAI Archives: looking to the future

Alberto Messina, Laurent BochRAI – Centre for Research and Technological Innovation

FIAT/IFTA World Conference 2015

Workshop

«Thinking the archives of 2020: opportunities, priorities, issues»

Vienna, Friday October 9th 2015

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

30

Summary

Current projects @ RAI archives

Looking to future documentation & access models

Storage for future archives – some thoughts

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

31

Master Digitisation

reads barcode,

verifies statuds

check-in

betacam, IMX,

16mm films, etc

REPO

hasRights?

hasCopies?

whichMaster?

hasPriority?

hasEditorialValue?

isMadeUpOfReels?

hasIdentfiers?

makes additional disaster

recovery copy over LTFS

(RAI open source tool)

orchestrates reception

of digitisation output

production & archive

media factory

T3transition-to-tapeless

delivery

Output Format = MXF/ D10

10x Automated Digitisation

of Betacam+IMX

tape cleaner

3x

IMX/EVTR

let‘s see if it’s

really okay!

QC

Output Formats: MXF/XDCAM HD422 25P

and HD – HiQual – 422 10bits

10x Manually operated Film

Scanning Station

Facts: 1300K beta+imx tapes; 800K film items; (before

selection, expected 20% after);

start by 2015Q4 with 3 Robotics; at full steam by

2016Q1; expected time of completion 5 years.

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

32

Rights Automation

Current issuesVarious different systems and practices

Need to check with narrative textual clauses

Weak links between Rights and AV-material

Too many manual processes & duplication

Legacy systems / lack of flexibility

Lack of reliable /detailed information on

reuse of archival excerpts in new

productions

ChallengesScope is Television, Radio, and Cinema

Handling the exploitation rights along their life-cycle

no ambiguity => no need to read again textual clauses

Continuously updating available rights in consequence of contract variants

consumption of runs / expiration

sales with exclusivity

Search & analysis on rights portfolio

Criteriaget all the people involved

accept idea of revising work-flows

simplify user interfaces as much as possible same interface might be integrated with multiple systems

give priority to input of new contracts

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

33

Rights Automation

Complete rights sheets, add

link to content

B

U

Y

E

R

R

I

G

H

T

S

U

S

E

R

rights clearance made

directly by the users

RaiChannelsRai Com Rai Web Rai Pubblicità

disclairmers &

constraints

rights sheet in

MPEG-21 MCO

Time line

Video Video clip 1 Video clip 2 Video Clip 3

Documentazione

metadata

Content 1 “Napoli prima e dopo i 4 giornate” di Aldo Zappalà Repertorio Ist. Luce

Diritti

metadata

Right 1 c.n. 1051801560; free tv, pacch italia, scad. 29.04.2012 AQ. Free tv, scad

31/12/2011

Content B

I

P

analysis

Links to administrative systems…

Benefits of adopting MPEG-21 MCOit supports the expression of contract condition in our scope.

Flexible, as it is possible to select the desired degree of generality / specificity; the various dimensions of the conditions can be combined in

all needed ways, for expressing the “reference exploitation rights” approved by the legal department.

Easy to extend and to integrate rights information with other domains.

Not just an organisational model, but a standard that can be widely used.

The Media Contract Ontology (MCO) is a standard of MPEG-21 multimedia framework. An electronic

format for machine readable contracts on media rights. The 1st edition was issued in 2013, the 2nd

edition is under ballot, exp. 2016.

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

34

Metadata Automation

Service-based integration of Automated Information Extraction tools

Genre categorisation

Machine translation

Spoken language identification

Automated sport content analysis

Quality analysis

Visual clustering and visual search

http://tosca-mp.euMaterial

selectionFeatures

generation

Test patterns

Features

selection

Learning/

Trimming

phase

Test

phase

Operational

phase

Training patterns

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

35

From media to semantics

Current metadata practises are mostly media-centric

You inspect or analyse the media

You document the media

You search for the media through metadata

Future approach: metadata (h)as value!

Exploiting semantics

Entities, relations, inferences

Media as one particular instance or realisation

Feed for new ways of making media business

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

36

Supporting technologies – Visual Search

Query

(e.g., from a camera, existing programme)

Visual search

Result

(e.g. from broadcast archive)

Visual Search

http://ict-bridget.eu

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

37

Storage

Archives in 2020 will face several challenges

Increase of content quality

Resolution, frame rate, dynamic range

Increase of distribution and publication channels

Digital TV

Internet

– Web is worth being archived

Mobile

– Apps are worth being archived

Digitization of archives

Millions of hours

Where to put resulting stuff?

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

38

Storage - AMS

Advanced Cloud Storage for Media

First efforts in VISION Cloud, FP7, 2010-2014

Key ideas

Computation near media

Content-based access rather than location-based

Rich metadata management at the storage level

Further development in collaboration with IBM in 2015

RAI MediaBridge

Archive application: metadata enrichment, digital preservation

processes, quality checks

Media Bridge Middleware

Media Project Management Interface

Projects Contents

EssenceRich

MetadataRelations

Compu-

tationalTasks

Identity Management

Interface

User Account

Access Policies

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

39

Conclusions

Digitise assets is only the first step

You have content safe

Make contract and rights managment efficient

You can use it

Exploitation of (meta)data & semantics

You can make new business

Storage (& computation) is a challenge

You can sustain the business

RAI Centro Ricerche e Innovazione Tecnologica – RAI Teche

40

Contacts

Alberto MESSINA

RAI – Radiotelevisione Italiana

Centre for Research and Technological Innovation

alberto.messina@rai.it

Laurent BOCH

RAI – Radiotelevisione Italiana

Centre for Research and Technological Innovation

laurent.boch@rai.it

Roberto ROSSETTO

RAI – Radiotelevisione Italiana

Teche

roberto.rossetto@rai.it

Task Force SRG for the archives 2020

42

SRG Archives strategy

Archiving of all the own production in its original quality

Standardisation, harmonisation and centralisation of the archive services

Automation

Opening and enhancement

43

Task Force for a realisation concept (roadmap)

5 main studies:

1. Archiving (9 month)

2. Searching interface (3 month)

3. Traceability (2 month)

4. Governance (4 month)

5. Finance (4 month)

44

Task Force for a realisation concept: archiving

Three steps:

1. What will be the SRG production in 2020 => Stakeholder program

departments, Benchmark

2. What will be the technical and structural evolution => Stakeholder operation

department, international Benchmark, Industry

3. What will be the archiving policy => Stakeholder Archives, Management

Conclusion

StorageNew quantitiesand densities

FormatsFor convergent use

Rightsmanagement

Traceability

Metadataautomation

User interface

Searching tool

WorkflowsIncludion right

formats andmetadata

Storage medium and

migration

Architecturone system or

services?

Rightsmanagement

FinancingBusiness model

OwnershipBroadcaster or

government

EnhancementThe new archivist

for Voting:http://etc.ch/z2Ic

Just for speakers

• http://directpoll.com/c?XDVhEt5xM0OLqhT9OlYuRt4ewVHWzKoU

Form Voting:

http://etc.ch/z2Ic

Resultat:

http://directpoll.com/r?XDbzPBd3ixYqg8XU1fxo04dELib3t8WBbAqGR5Y7f