EE616 Technical Project Video Hosting Architecture By Phillip Sutton.
-
Upload
judith-marsh -
Category
Documents
-
view
218 -
download
0
Transcript of EE616 Technical Project Video Hosting Architecture By Phillip Sutton.
![Page 1: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/1.jpg)
EE616 Technical Project
Video Hosting Architecture
By Phillip Sutton
![Page 2: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/2.jpg)
Problem Description
• Need to store and serve massive amounts of video data.
• Solution must be:– Scalable– Reliable– Relatively fast
![Page 3: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/3.jpg)
Complications
• Oh yeah….
• Have relatively little cash.
• SO, need minimal startup costs!
![Page 4: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/4.jpg)
Options
• YouTube…Believe it or Not.
• Build it yourself.
• Managed or dedicated hosting
• Content Delivery Network (CDN).
• Amazon Simple Storage Service.
![Page 5: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/5.jpg)
YouTube
• Free to use.
• 100 million videos served daily.
• Hosted on Google’s reliable and scalable infrastructure.
![Page 6: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/6.jpg)
Video Sharing Site ComparisonWebsite YouTube Yahoo Video Veoh Vimeo
Unique Visitors per year 205,593,000 48,026,000 11,476,000 569,000
Max Video Bit Rate (kbps) ~2001 3003 1,500 1,600
Max Upload File Size (mb) 1002 150 250 500/wk
Max Length (min) 10 N/A N/A N/A
Max Screen Size(s) 320x240 320x240 640x480 1280x7204
Host Format (streaming) FLV FLV FLV FLV
Processing Time Up to several hours
Up to several hours
Few hours Minutes5
1 estimated 2 increasing to 1 GB 3 upcoming 700 kbps 4 claims this capability
![Page 7: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/7.jpg)
Drawbacks
• Limited file size– Need 4.7 GB.
• Limited bitrate– Implies relatively low quality.
• For higher bitrate sites– Still suffer from limited file size.
• No real options to manage library.
• No real options to monetize.
![Page 8: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/8.jpg)
Build It Yourself
• Have almost complete and utter control.
• No messy CDN contracts to deal with.
• Scalable; depending on your budget.
![Page 9: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/9.jpg)
Drawbacks
• Expensive to start.
• Expensive to grow.
• Requires space, power, and resources.
• Requires knowledgeable manpower to maintain and support.
![Page 10: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/10.jpg)
Drawbacks
http://www.acadweb.wwu.edu/dbrunner/P7040180.JPG
![Page 11: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/11.jpg)
Managed/Dedicated Hosting
• Let someone else deal with it – setup, maintenance, and support.
• Mostly reliable– Many claim 99.9% uptime.
• Affordable to start– 500 GB of storage and 2,500 GB bandwidth .– cost about same as small efficiency apartment
on Southside.
![Page 12: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/12.jpg)
Drawbacks
• Can’t scale with you.
• Overage costs will get you!!!
• Can’t control hardware.
• Can’t make favorable networking agreements.
![Page 13: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/13.jpg)
Content Delivery Networks
• Multiple data centers.
• Most have direct internet backbone access.
• Designed for performance.
• Replicate content .
![Page 14: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/14.jpg)
Drawbacks
• Traditionally marketed to enterprises– Apple iTunes uses Akamai.
• Hard to figure costs w/o signing agreement.
• Prepay for chunks of storage and bandwidth.
• Exceeding allocation can be costly.
• Pay for idle storage and bandwidth.
![Page 15: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/15.jpg)
Amazon Simple Storage Service
• New kid on the block.
• Same infrastructure as Amazon.com– Scalable, high availability, low latency.
• Unlimited storage.
• Unlimited bandwidth.
• Pay only for what you use.
• No contracts; zero cost to startup.
![Page 16: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/16.jpg)
Drawbacks
• New kid on the block.
• Latency perhaps not as good as CDNs.
• Bandwidth costs may still be an issue.
• No server side processing.
![Page 17: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/17.jpg)
Comparing Costs
• Build library of 5000 4.7 GB DVDs• Deliver 100TB per month.
Hosted CDN Amazon S3
Storage $241,000 $23,552 $3,523
Bandwidth $141,312 $29,696 $15,153
Total Per Month $382,312 $53,248 $18,676
![Page 18: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/18.jpg)
S3 Overview
• Store objects up to 5 GB in size with metadata.
• Objects stored in buckets.
• Unlimited number of objects per bucket.
• Each bucket is owned by an Amazon Web Service (AWS) account.
![Page 19: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/19.jpg)
S3 Overview
• Object is identified unique key.
• Use REST-style HTTP, SOAP, or HTTP GET/PUT interfaces.
• Supports BitTorrent protocol.
• Authorize requests with ACLs.
![Page 20: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/20.jpg)
S3 Overview
• Authenticated URLs can be created with time-bounded validity.
![Page 21: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/21.jpg)
Over Simplified Architecture
Web Server /
CMS
S3Web
Client
![Page 22: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/22.jpg)
Over Simplified Architecture
• Use S3’s online storage service and economy hosting/bandwidth provider.
• Use a content management system to track all assets stored on S3.
• Web client communicates with CMS and S3.
![Page 23: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/23.jpg)
Upload Content
• Web client requests authentication keys from CMS.
• Once keys are received, client can send files directly to S3.
• Or send files to CMS without access keys.
• Then CMS forwards to S3.
![Page 24: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/24.jpg)
Get Content
• Web client request content from CMS.
• CMS issues authenticated URL with limited time to live.
• Client then has preset amount of time to retrieve file directly from S3.
![Page 25: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/25.jpg)
Issues
• In addition to drawbacks mentioned earlier
• No server-side processing of scripts.
• Need to better handle read/write failures.
• Need to build your own software.
![Page 26: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/26.jpg)
Next
• Lot’s of work left to do.
• Create more detailed architecture.
• Work out code details.
• Implement and test scalability and performance.
![Page 27: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/27.jpg)
Future
• Integration with content management system
• Integrate with Amazon’s EC2 service.
• Explore BitTorrent protocol for increased through-put.
![Page 28: EE616 Technical Project Video Hosting Architecture By Phillip Sutton.](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649e265503460f94b15cf2/html5/thumbnails/28.jpg)
QUESTIONS