Projet AAP FUI11

The NExt Video Experience

Project short presentation

Catherine Serré – Technicolor R&D France 2012 June 12

Project descriptionFrench collaborative project (labelled by «Images et réseaux» and «Cap Digital» clusters)

Duration 30 monthsFrom 2011-October-01 to 2014-March-31

Subject: HDR video for broadcasting

Two main goals:Generate HDR video content

Explore all the bricks of a HDR video distribution chain from content creation, through distribution, to visualization by the user

NEVEx 32012/06/12

Project description

NEVEx 42012/06/12

CONTENT CREATION

DISTRIBUTION

Encoding/ TransportFormat /adaptation

CGI content

Real scene capture

RENDERING / DISPLAY

Decoding / Rendering

Display

Binocle(real scene capture)

Polymorph Soft. /IRISA

(CGI content)

Technicolor(compression)

Thomson Video Network

(compression)

Technicolor(compositing,

rendering)

IRISA(rendering)

IRCCYN(quality

assessment)

DxO(compositing,

rendering)

DxO(content processing)

Télécom ParisTech(content conversion)

Télécom ParisTech(compression)

Transvideo(display)

TF1(Broadcaster)

AccepTV(quality

assessment)

Content creation: Real scene captureThe idea is to :

reuse the Binocle 3D rig with a null baseline (aligned views) Use an appropriate neutral gray filter on one camera in order to generate two views of the same scene: an under-exposed view and an over-exposed viewcorrect the geometric disparity between images (due to sensor, optics … disparities) with Binocle modified disparity killer applicationPerform the fusion of the two views to generate a HDR sequence

Challenges:Research companies / academics and production companies don’t speak the same language nor share the same vocabularyNeed camera and workflow characterization to produce linear contentFusion algorithm have to be robust to video characteristics (temporal luminance stability, robustness to flashes …)How much dynamic range increase (f-stop) is possible?

NEVEx 52012/06/12

Content creation: CGI contentCreate HDR CGI video sequences

Generate different kinds of CG sequences which make tone mappers fail

Use global illumination engines: physical luminances

Control of the luminance and gradient levels in the CG sequences by changing:

Light sourceMaterialGeometryCamera position

Determine which tone mappers are relevant for each class of CG sequence

NEVEx 62012/06/12

HDR video processingVideo fusion:

Required for real content creation from two LDR viewsChallenges:

Temporal aspect: luminance stability over time, robustness to flashes …Interactivity:

Semi-automatic for post-production to preserve artistic intentAutomatic for end device (distribution solution 2)

Video Tone Mapping:Required for LDR display backward compatibilityChallenges:

Temporal aspect: luminance stability over time, robustness to flashes …Real-Time implementation at decoding stage and near real-time for post-productionInteractivity:

Semi-automatic for post-production to preserve artistic intentAutomatic for end device (distribution solution 2). Guided Tone Mapping ?

Inverse Tone Mapping:Short-term: alternative method to produce HDR video contentLong-term: Conversion of legacy content into HDR for HDR ditribution chain

NEVEx 72012/06/12

HDR video quality assessmentSubjective video quality evaluation for different bricks of the chain:

FusionTone MappingInverse Tone MappingCompressionFormat representation…

Development of an objective HDR video quality metric

NEVEx 82012/06/12

Solution 1: reuse legacy infrastructure and demonstrate the benefit of HDR technology right now

Pros: legacy infrastructure as isCons: dedicated to LDR displays

Solution 2: reuse legacy equipments and build an end-to-end HDR video chainPros:

LDR and HDR displays addressedlegacy equipments

Cons: Native HDR not addressedlimited to 2 views (quality ?)processing power at decoding stage

Challenge: video fusion and TM at decoding stage

Solution 3: long-term native HDR video chainPros:

EfficiencyBackward compatibility

Cons:New equipment => new standard

Challenges:New HDR representation: float ? Integer ?Reuse existing integer based compression techniques (MPEG …)?LDR backward compatibility

Distribution / Demonstrators

NEVEx 92012/06/12