23-04-19 Martin Bisson 1

Stereoscopic video and GStreamer

By Martin Bisson

Presentation outline

IntroductionUse casesImplementationRoad ahead

23-04-19 Martin Bisson 2

Presentation outline

IntroductionUse casesImplementationRoad ahead

23-04-19 Martin Bisson 3

Stereoscopy

23-04-19 Martin Bisson 4

Recent popularity

3D-ready hardware

3D moviesAvatar => ~2.8 billion USD

23-04-19 Martin Bisson 5

Presentation outline

IntroductionUse casesImplementationRoad ahead

23-04-19 Martin Bisson 6

Use cases

1) Merge 2 video streams into a stereo stream

2) Convert a (double) video stream into a stereo stream

23-04-19 Martin Bisson 7

Use cases

3) Convert a stereo stream into a « regular » video streamCan be displayed or encoded by normal plugins

4) Generate an anaglyph video stream from a stereo stream

5) Output a stereo stream on various 3D display hardware

23-04-19 Martin Bisson 8

Presentation outline

IntroductionUse casesImplementationRoad ahead

23-04-19 Martin Bisson 9

Implementation

What is a « stereo stream »?Buffer flags?New caps

video/x-raw-{rgb,yuv,gray}-stereolayout = { 1122 , 1212 }fourcc = { YUY2, YVYU, UYVY , Y800 , GREY , Y8 , Y16 }

23-04-19 Martin Bisson 10

Memory layout

23-04-19 Martin Bisson 11

1122Memory-consecutive

1212Row-interleaved

Supported formats

Only packed, no planarRGB : RGBx, xRGB, BGRx, xBGR, RGB,

BGR, RGB_16, RGB_15YUV : YUY2, YVYU, UYVY, Y800,

GREY, Y8, Y16GRAY : GRAY8, GRAY16

23-04-19 Martin Bisson 12

Plugins

video3dmergeGstCollectPadsmode : memory-consecutive or row-interleaved

23-04-19 Martin Bisson 13

Plugins

video3dconvertGstVideoFiltermode : left-right or top-bottom

23-04-19 Martin Bisson 14

Plugins

video3dpresentGstVideoFiltermode : a lot of them…

23-04-19 Martin Bisson 15

video3dpresent modes

Left, RightLeft/Right, Right/Left, Top/Bottom, Bottom/TopAnaglyph

Gray/Gray, Gray/Color, Color/Gray, Color/ColorRed/Cyan, Green/Magenta, Blue/Yellow, and vice-versa

23-04-19 Martin Bisson 16

Anaglyph implementation

« C templates »Inspired by ffmpegcolorspace (imgconvert.c)

Orc optimizationOnly RGBx/BGRx/xRGB/xBGR output formatsYUY2/YVYU/UYVY/Y800/GREY/Y8/Y16 input

formats for gray-gray anaglyphRGBx/BGRx/xRGB/xBGR input formats for all

anaglyph types

23-04-19 Martin Bisson 17

23-04-19 Martin Bisson 18

Example application

23-04-19 Martin Bisson 19

Presentation outline

IntroductionUse casesImplementationRoad ahead

23-04-19 Martin Bisson 20

Integration

Not in main GStreamer tree (in a branch at http://www.gitorious.org/video3d/video3d/)

Talks about how to handle stereo video:Buffer flagsMetadataNew caps

23-04-19 Martin Bisson 21

Integration

New caps (video/x-raw-*-stereo)Caps negotiation and all that jazz…

Buffer flags/metadataBuffers can be used in existing plugins

23-04-19 Martin Bisson 22

Integration

23-04-19 Martin Bisson 23

3D hardware

Implement stereo sinksNVIDIA 3D VisionQuad Buffered OpenGLiZ3DEtc…

See Stereoscopic Player (http://www.3dtv.at/)

23-04-19 Martin Bisson 24

Stereo codecs

Multiview Video Coding (H.264/MPEG-4 AVC)Uses temporal AND inter-view predictionExisting implementation by Nokia

http://research.nokia.com/page/4988

23-04-19 Martin Bisson 25

23-04-19 Martin Bisson 26

Questions?