Direct Access Video Streaming Scheme Supporting Effective Error ConcealmentChun-Ming Huang, Kai-Chao Yang and Jia-Shung Wang

Outline

Introduction Hierarchical Coding Scheme Center-weighted Hierarchical Coding Scheme Proximity Approximation Questions in the Hierarchical Structure Modified Sequential Coding Scheme Decoder Error Concealment Error Concealment in Hierarchical Coding Structure Error Concealment in Modified Sequential Coding

Structure Conclusion and Future Work

Introduction

VCR OperationStop-Resume, Pause-Resume, Slow motion,

Jump, Fast Forward Scan, Back Forward Scan

Group of Pictures (GOPs)

I B B P B B P B B P

1 3 4 2 6 7 5 9 10 8

I P P P

GOP GOP

Introduction

General sequential video coding system

4

2

01

3

the average number of frames to be transmitted for randomly accessing an arbitrary frame

time

2

1...21 1

n

n

i

n

n

n

i

…

I

P

P

P

P

Hierarchical Coding Scheme

6 8 10420 12 145 91 13113 7

I P P PP PP

GOP

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14time

1lg2

1lg

2lg...23221 1lg2

nn

n

n

n n

Improve the efficiency of random access operation.

(n -> lg n)

Center-weighted Hierarchical Coding Scheme

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

6 8 104

2

0

12

14

5 9

1 13

113

7

6 8 10420 12 145 91 13113 7

6 8 10420 12 145 91 13113 7

3lg23

)1(lg211lg

1

n

nnn

inn

i

i

Center-weighted Hierarchical Coding Scheme e.g.

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

6 8 104

2

0

12

14

5 9

1 13

113

7

x2

Cost : 7 Cost : 7

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

6 8 104

2

0

12

14

5 9

1 13

113

7

Cost : 10 Cost : 8

x3

0 21 3 4 5 6 8 10 12 147 9 11 12 13

Proximity Approximation

Human visual system is normally not very sensitive to minor change of scenes in the temporal domain. (e.g. 0, 3, 6 vs. 0, 4, 6)

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

6 8 104

2

0

12

14

5 9

1 13

113

7

Cost : 6 Cost : 6

x3

Questions in the Hierarchical Structure More overhead while sequential playing The necessity of backward playback? Fast forward speed x3, x5, x7, … are quite

similar to x2, x4, x6, …

7 73

1

73

0 21

73

73

73

3

7 7 7 7 11

54

5

5 61 2 4

7

119

11

810

13

1212

9

13

10 11

14

12 13 14

0 1 2

0 21

3 4 5

3

6 7 8 10

4 5 6 7

12 13

8 9 10 11 12 13 14

9 11

1114

time

56

611

119

98

1013

1314 Initial delay &

Large buffer

Modified Sequential Coding Scheme Add redundant links to sequential coding system

0 21 3 4 5 6 8 10 12 147 9 11 12 13

0

0 2

0 2 4 6 8 10

4 6 8 10 12 14

2 4 6 8 10

7 73

73

51

73

7

3 9

11

11 13

13

7

11

0

0 4

4 8

8 12

59

1113

7 73

73

51

7 7

9

1111 9

11

13

x2 x4time time

Decoder Error Concealment

Recovery of Texture Information Syntax-Based Repair Recovery of Motion Vectors

Set to zero Use the corresponding MVs in the previous frame Use the average MVs from spatially adjacent MBs Use the median MVs from spatially adjacent MBs Re-estimate the MVs

Error Concealment in Hierarchical Coding Structure

6 8 10420 12 145 91 13113 7

0 21 3 4 5 6 8 10 12 147 9 11 12 13

7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

Error Concealment in Hierarchical Coding Structure Add redundant links into the hierarchical

tree.7

6 8

5 9

10

11

4

3

2

1

0 12

13

14

Each damaged leaf node can be recovered via interpolation approach.Each damaged internal node can be reconstructed through its son or the redundant link.※Either the redundant link or original link must survive.

6 8 104

2

0

12

14

5 9

1 13

113

7

Error Concealment in Modified Sequential Coding Structure

0 21 3 4 5 6 8 10 12 147 9 11 12 13X X X X

Conclusion and Future Work

The proposed video streaming schemes support direct access and error concealment.

Future WorkReduce the buffer size.Reduce the number of the redundant links.Reduce the initial waiting time in the

hierarchical coding structure.