1
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
How to synthesize temporal visual signals as transient input traffic for discrete event simulation?
Mauritz Panggabean and Leif Arne RønningenDepartment of Telematics, NTNU
3rd International Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) 2011Budapest, Hungary, 5-7 October 2011
Synthesizing Transient Traffic of Temporal Visual Signals for Discrete Event Simulation
2
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Outline
• Introduction– Background and vision
– Framework: the big picture
– Why transient traffic?
• Preparation and experiment– Type of input: the scenes
– Processing the clips
– Modeling using linear and power functions
• Results: synthetic transient traffic using power function
• Conclusion and the next work
3
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Background and vision
• Futuristic tele-immersive environment system for real-time artistic collaborations with near-natural quality– Arrays of auto-stereoscopic 3D displays and high-end cameras on all
surfaces of the collaboration space
– More than one performers in a collaboration space
– Multiview with view activated by the gaze of the performers
• Challenge: modeling such environment in discrete event simulation prior to construction– Simulation for feasibility study with (un)compressed input data
• First step: a model for synthetic transient input traffic from human motion and camera modes
Begin with the end in mind.Stephen Covey
4
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Framework: the big picture
If everyone is thinking alike, then somebody isn't thinking.George S. Patton
Synthesis of multiobject
human motion
Uncompressed visual traffic: Multiview
projected body-areas based on eye gazes
Incorporating effects from
image compression
Compressed visual traffic: Multiview projected body-areas based on eye gazes
Discrete-event simulation of complex multiview multiobject distributed tele-
immersive collaboration system
Synthesis of eye gazes
Humanoid model
Stochastic and deterministic human motion
Formal model
Synthetic multiobject human
body motions
Synthetic eye gazes
Computation of projected areas based on eye gazes
Rate-distortion curves of image compression
5
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Why transient traffic?
• Stationary traffic nullified by interframe compression
• Important: transient slopes, variations, extreme values and duration (not in mean values)
• Piecewise analysis: a general formal model of a few parameters to synthesize each piece as input traffic (GOAL)
Have you got a problem? Do what you can where you are with what you've got.Theodore Roosevelt
6
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Types of input: the scenes
• Only one static camera, one view and one object• Three scenes as input video clips:
1. Clip PANNING• Object enters the scene from left and disappears on the right side• Equivalent to camera panning
2. Clip ZOOM• Object gradually moves closer to the camera • Equivalent to camera zoom
3. Clip MOTION• The object performs some motions with the limbs at the center of the
scene
Everything should be made as simple as possible, but not simpler.Albert Einstein
7
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Processing the clips
It is an acknowledged truth in philosophy that a just theory will always be confirmed by experiment. Thomas Malthus
TranscodingDe-interlacing
Original color clips (1920x1080, 30Hz)
AVI color clips (1280x720, 30Hz)
Objectsegmentation
Uncompressed input traffic
- Off-the-shelf camera- Interlaced, uncompressed- One person- Uniform color background
- Matlab- PC (2.99GHz, 8.0 GB RAM)
- Object only (the person)- 8 bits per pixel- RGB channels
8
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Input traffic from the clips
No amount of experimentation can ever prove me right; a single experiment can prove me wrong. Albert Einstein
20 25 30 35 40 45
1 50 100 150 200 250
1 25 55 118 127 145
9
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Modelling as power function
In much of society, research means to investigate something you do not know or understand. Neil Armstrong
Power function f(x) = axb as the general model for synthesis
a = 1.108, b = 0.8518RMSE = 0.09655
c = 1.116, d = 0.03048RMSE = 0.104
Power function f(x) = axb and linear function f(x) = cx + d with normalized frame number
a = 0.9533, b = 1.339RMSE = 0.03234
c = 0.9854, d = -0.08577RMSE = 0.04202
a = 1.134, b = 0.8762RMSE = 0.1677
c = 1.145, d = 0.02072RMSE = 0.172
10
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Research is to see what everybody else has seen, and to think what nobody else has thought.Albert Szent-Gyorgyi
Power function for transient input traffic synthesis in simulation• Transient traffic with increasing trend
f(x) = axb + d + e• Transient traffic with decreasing trend
f(x) = a(1-xb) + d + e
• Parameters:– b, to control the curve bending: 0.5 < b < 1
– a = Dmax – Dmin, d = Dmin: Dmin and Dmax as the min/max data
– e, to control the random smoothness of the curve at point x.
emin < e < emax is uniformly distributed, emin ≤ emax and e < a.
– Frame rate F and simulation time S (sec): t = 1/F and x = t/S
11
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Synthetic transient input traffic
An experiment disproving a prediction is a discovery. -Enrico Fermi
Panninga = 4b = 0.65c = 0F = 19
Zooma = 12b = 1.34c = 8F = 230
MotionIncreasing parta = 3, b = 0.75, c = 7F = 9
Decreasing parta = 2.5, b = 2.5, c = 7.5F = 11
For allemin = 0 emax = 0.5S = 1
Adjacent pieces of arbitrary transient traffic connected by setting their values of a and c
12
3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011
Conclusion and the next work
Research is what I'm doing when I don't know what I'm doing. -Wernher von Braun
• Power function as a general formal model for transient traffic synthesis as input for discrete event simulation
• Covers human motion, camera panning and zoom for one view
Synthesis of multiobject
human motion
Uncompressed visual traffic: Multiview
projected body-areas based on eye gazes
Incorporating effects from
image compression
Compressed visual traffic: Multiview projected body-areas based on eye gazes
Discrete-event simulation of complex multiview multiobject distributed tele-
immersive collaboration system
Synthesis of eye gazes
Humanoid model
Stochastic and deterministic human motion
Formal model
Synthetic multiobject human
body motions
Synthetic eye gazes
Computation of projected areas based on eye gazes
Rate-distortion curves of image compression
Top Related