.
The Big Picture: YALMS
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 2/17...
2/17
.
YALMS: Yet Another LMS• feature: input, processing, and delivery in equal shares 17
Step 1: Record and Stream
V A
S Creator
Viewers
Content Server
HTML5
HTML5 Muxer
Record Screen Streaming
Step 2: Annotate
Lecturer
S
Auto- annotate
C
17 M.Zhanikeev+1 "YALMS: Yet Another Lecture Management System..." IEICE・ET研 (July 2013)M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 3/17...
3/17
.
A Business Day at YALMS
• many screen and video streams uploaded by users 15
• many concurrent video annotation tasks (PDF + VIDEO) 16
• constantly changing popularity distribution of content, needs
dynamic relocations
15 "YALMS Project Home" https://github.com/maratishe/yalms (June 2013)
16 "YALMS Annotation Robot (software behind this paper)" https://github.com/maratishe/yalms.annotation (September 2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 4/17...
4/17
.
YALMS Topics
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 5/17...
5/17
.
YALMS: The Big SSD Question
.The Question is..
.... can such a rich multimedia architecture benefit fromSSDs?• WHY1: SSDs have properties different from those of HDDs
◦ read/write, random/sequential
• WHY2: SSDs are becoming cheaper (MLC) so why not to use them?
• WHY3: parallel/multicore processing is not kind to HDDs but SSDscan cope
• WHY4: 50Mbytes/s throughput is not enough any more
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 6/17...
6/17
.
Where Are We Now? ExistingResearch
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 7/17...
7/17
.
Existing Research (1)• roughly, two groups:
1. SSD versus HDD research• filesystem abstraction and scheduling -- load balancing between SSD and HDD 01• SSD as cache memory 0203
2. parallel processing and themulticore• job graph partitioning 18• local Big Data processing 19• multicore streaming 11-- 14
01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)
03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)
18 X.Sui+3 "Parallel Graph Partitioning on Multicore Architectures" 23rd LCPC (2010)
19 R.Chen+2 "Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications on Multicore..." 19th PACT (2010)
11 Y.Wei+3 "Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-Core Processors" ACM SAC (2010)
14 V.Galiano+3 "SSD architecture such as multi-channel and multi-way" EURASIP Journal ... Signal Processing (2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 8/17...
8/17
.
Existing Research: What is Missing?
.In a Nutshell, Existing Research is.....
.
• extremely specialized and is built for one application only
• ... or, in business, the use is very primitive -- a temporary memory buffer
• ... at the same time.. YALMS needs generic load balancing◦ one design fits all
• changes along the timeline should cause changes -- dynamic optimization• ...
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 9/17...
9/17
.
The YALMS.hiperf project
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 10/17...
10/17
.
Meet the YALMLS.hiperf Project
Multimedia System
Creators
Viewers
Manage
HDDs HDDs HDDs HDDs HDDs SSDs
Cores Cores Cores
• a pool of SSDs,HDDs, etc.
• a separate pool ofCPU cores
• steady arrival anddeparture rates oftasks/processes
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 11/17...
11/17
.
YALMS.hiperf : The Overall Design
YALMS.hiperf
YALMS.hiperfRobot
A Multimedia System
ProcessProcessProcessProcessProcessProcessProcessReq(details)
Rep(agent)
AgentsAgentsAgentsAgentsAgentsHDDHDDHDDHDDHDDs
SSDsSSDsSSDsSSDsSSDsSSDs ….
• a firmware approach
• tasks/processes sendrequests
• replies point to agents
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 12/17...
12/17
.
YALMS.hiperf : The Basic Unit
…
Proc
essin
g
logic
SSD
HDD
ENGINE
Network storage
Input Next step
Read/write? Seq./rand.? (Chunk) size
HiPerf/Normal? Longer plans?
Multicore
Fork Join
• traditional parallelprocessingformulation
• engine performsruntimeoptimization
• tasks are assignedclasses
01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 13/17...
13/17
.
YALMS.hiperf : Runtime Optimization.Basic Formulation..
.
start from zero, collect history, thenoptimize accordingly • tasks should naturally have the few
heavy hitters distribution◦ if not, tasks should be partitioned
• then simply
minimize∑
i∈tasks
∑j∈time
lagi,j (1)
• ... details in future publications
03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 14/17...
14/17
.
Wrapup
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 15/17...
15/17
.
How Far is The Project?• software for multicore video annotation on top of SSD
◦ mostly reads, very few writes
• currently working on the generic version of software, will release soon
screen video
SlidesPDF
LaTeX/BeamerKeynote
PowerPoint…
VLCffmpeg
I-Frames I-Frames
pages PNGs
ANNOTATION
slices
PNGstimestamps
End productscenes cleanup
splitpdf2png
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 16/17...
16/17
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[01] J.Kim+4 (2009)Disk Schedulers for Solid State Drives7th ACM EMSOFT
[02] S.Rizvi+1 (2010)Flash SSD vs HDD: High Performance .... Multimedia Storage Systems2nd ICCET
[03] Y.Lee+1 (2013)Mapping granularity and performance tradeoffs for solid state driveThe Journal of Supercomputing
[04] (2012)Data Compression in the Intel Solid-State Drive 520 SeriesIntel Technology Brief
[05] (2013)Cisco Content Delivery Engine 250Cisco Data Sheet
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[06] B.Sajadi+3 (2011)...SSDs for Large-Scale Interactive Graphics ApplicationsSymp. on 3D Graphics and Games (I3D)
[07] D.Zhao+1 (2013)HyCache: a User-Level Caching Middleware for Distributed File SystemsACM HPDC
[08] M.Ryu+2 (2011)Impact of Flash Memory on Video-on-Demand Storage: Analysis of Tradeoffs2nd ACM MMSys
[09] (2013)Performance ... On DemandConcurrent Computer Corporation (STEC), Case Study
[10] M.Ryu+2 (2012)Why ...flash-based multi-tiered storage systems performing poorly for HTTP videostreaming?22nd NOSSDAV
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[11] Y.Wei+3 (2010)Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-CoreProcessorsACM SAC
[12] S.Sankaraiah+3 (2011)GOP Level Parallelism on H.264 Video Encoder for Multicore ArchitectureIPCSIT
[13] T.Dias+2 (2010)H.264/AVC framework for multi-core embedded video encodersSoC
[14] V.Galiano+3 (2013)SSD architecture such as multi-channel and multi-wayEURASIP Journal ... Signal Processing
[15] (June 2013)YALMS Project Homehttps://github.com/maratishe/yalms
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[16] (September 2013)YALMS Annotation Robot (software behind this paper)https://github.com/maratishe/yalms.annotation
[17] M.Zhanikeev+1 (July 2013)YALMS: Yet Another Lecture Management System...IEICE・ET研
[18] X.Sui+3 (2010)Parallel Graph Partitioning on Multicore Architectures23rd LCPC
[19] R.Chen+2 (2010)Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications onMulticore...19th PACT
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
Top Related