Resumable File Upload API using GridFS and TUS
29
MONGO DB FOR CAT LOVERS
-
Upload
khangtoh -
Category
Technology
-
view
1.807 -
download
4
description
TUS is a resumable file upload protocol and with MongoDB GridFS, we build an API for uploading files through a REST API and show how to scale this API horizontally using MongoDB as the storage for these files. Singapore MongoDB User Group March Meetup
Transcript of Resumable File Upload API using GridFS and TUS
MONGO DB FOR CAT LOVERS
About me• Developer / Entrepreneur
• Co-Founder / CTO at PicoCandy Singapore and MogiMe Inc, San Francisco :)
What do you do as an entrepreneur?
You think of ideas
You see funny cats pics at work
Sometimes you get bored and google for funny cats at
work
And when you suddenly thought of this awesome idea,
you can’t sleep
What if !!??!!User Generated Funny Cat
Pics and Videos!
Shot using their mobile phone camera!
Using the Catstagram app!
MVP Development 101
• Wrote ruby server code
• Wrote iOS code
• Wrote ruby server code
• Wrote iOS code
Problem
How to do file upload on mobile?
Why File Upload for mobile sucks*
Why File Upload for mobile sucks*
Why File Upload for mobile sucks*
SUCKY SG MOBILE NETWORK
How to make file upload RESUMABLE for mobile?
Then you google like every developer should*
TUS.IO a resumable file upload protocol on top of HTTP
What is the TUS protocol?
• Simple, open and free
• HTTP-based ( POST, HEAD, PATCH )
• Split file into chunks
• Send smaller pieces, server keeps track of chunks / offset
• Web and mobile clients library
What is the TUS protocol?
POST "/files" HTTP/1.1Host: tus.example.orgContent-Length: 0Final-Length: 100
RESPONSE:
HTTP/1.1 201 CreatedLocation: http://tus.example.org/files/1
What is the TUS protocol?
PATCH "/files/1" HTTP/1.1Host: tus.example.orgContent-Length: 70Offset: 0[file data]
RESPONSE:
HTTP/1.1 200 Ok
What is the TUS protocol?
HEAD "/files/1" HTTP/1.1Host: tus.example.org
RESPONSE:
HTTP/1.1 200 OkOffset: 70
What is the TUS protocol?
PATCH "/files/1" HTTP/1.1Host: tus.example.orgContent-Length: 30Offset: 70
[remaining file data]
RESPONSE:
HTTP/1.1 200 Ok
Awesome! What about the servers and the
clients?• tusd - reference server implementation
using Go
• tus-ios-client, native objective-C client library
• tus-jquery, javascript client library
• rubytus, ruby gemhttps://github.com/picocandy/rubytus
Where are the chunks?
• tusd - reference server implementation using Go
• tus-ios-client, native objective-C client library
• tus-jquery, javascript client library
• rubytus, ruby gemhttps://github.com/picocandy/rubytus
Where are the chunks? Take 2.
What is GridFS?
• GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16MB.
• Just like the other collections
• By default, GridFS uses two collections with names prefixed by fs bucket:
• fs.files and fs.chunks
• Use a different bucket name? Yes!Create multiple buckets in a single database? Oh yes!
What is GridFS?
{
"_id" : <ObjectId>,
"length" : <num>,
"chunkSize" : <num>
"uploadDate" : <timestamp>
"md5" : <hash>
"filename" : <string>,
"contentType" : <string>,
"aliases" : <string array>,
"metadata" : <dataObject>,
}
The files Collection - fs.files
What is GridFS?
{
"_id" : <ObjectId>,
"files_id" : <ObjectId>,
"n" : <num>,
"data" : <binary>
}
The chunks Collection - fs.chunks
Mongo Ruby Driverrequire 'mongo'include Mongo
@db = MongoClient.new('localhost').db('picotusd')@grid = Grid.new(@db)
Saving Fileimage = File.open(“cat.jpg")
id_for_cat = @grid.put(image, :filename => “cat.jpg")
Retrieving Filethe_first_cat_image = @grid.get(id_for_cat)
Demo / Code
Questions?
