Visualizing Mobile Broadband with MongoDB

29
Visualizing Mobile Broadband Federal Communications Commission Eric Spry, Acting Geographic Information Ocer

description

The FCC Speed Test mobile app is a crowdsource project on mobile network performance, collecting millions of samples per month. In order to make the collected information valuable to consumers, a new approach was needed to present this rich resource in timely and intuitive manner. The Visualizing Mobile Broadband project is a web application that presents mobile network information using maps and interactive metrics. Our small team of developers turned to MongoDB for a solution that is flexible and scalable as volume increases and needs evolve. We present our project from conceptual framework to working prototype and highlight organizational obstacles, geospatial and technical issues encountered along the way.

Transcript of Visualizing Mobile Broadband with MongoDB

Page 1: Visualizing Mobile Broadband with MongoDB

Visualizing Mobile BroadbandFederal Communications CommissionEric Spry, Acting Geographic Information Officer

Page 2: Visualizing Mobile Broadband with MongoDB

Measuring Broadband America

Fixed: 12,000 participants

Mobile: nearly 100,000 installs for Android, and 50,000 for iOS

www.fcc.gov/measuring-broadband-america

Page 3: Visualizing Mobile Broadband with MongoDB

Download This Appavailable in Google Play and iTunes App Store

Page 4: Visualizing Mobile Broadband with MongoDB

"I like the idea of this app but it's not that useful when I can only see my data. I'm waiting to see if they release their heat maps & infographics of carrier performance in Nov 2014 like they promised... " -- Google Play Store Comment

Page 5: Visualizing Mobile Broadband with MongoDB

Visualizing Mobile Broadband

Page 6: Visualizing Mobile Broadband with MongoDB

How the Gov't normally releases info

Page 7: Visualizing Mobile Broadband with MongoDB

There is a better way

Page 8: Visualizing Mobile Broadband with MongoDB

Feeling Spatial

Page 9: Visualizing Mobile Broadband with MongoDB
Page 10: Visualizing Mobile Broadband with MongoDB

xkcd on Heat Maps

Page 11: Visualizing Mobile Broadband with MongoDB

Hex Binning

Page 12: Visualizing Mobile Broadband with MongoDB

Some early examples

Page 13: Visualizing Mobile Broadband with MongoDB

Later Versions

(based on early draft data)

Page 14: Visualizing Mobile Broadband with MongoDB

SQL approach was good for early investigation, but not for delivering a useful product to consumers.

Enter MongoDB

Page 15: Visualizing Mobile Broadband with MongoDB

Architecting the System

Page 16: Visualizing Mobile Broadband with MongoDB

MongoDB

"tests" : [ { "type" : "JHTTPGETMT", "timestamp" : "1398965279", "warmup_time" : "5323371", "target_ipaddress" : "4.71.185.62", "success" : "true", "number_of_threads" : "3", "transfer_bytes" : "6813952", "bytes_sec" : "453803", "datetime" : "2014-05-01T17:27:59+0000", "target" : "samknows1.nyc2.level3.net", "warmup_bytes" : "2627029", "transfer_time" : "15015192" }, { "type" : "JHTTPPOSTMT", "timestamp" : "1398965305", "warmup_time" : "5569560", "target_ipaddress" : "4.71.185.62", "success" : "true", "number_of_threads" : "3", "transfer_bytes" : "3569696", "bytes_sec" : "227745", "datetime" : "2014-05-01T17:28:25+0000", "target" : "samknows1.nyc2.level3.net", "warmup_bytes" : "1313376", "transfer_time" : "15674058" },

Page 17: Visualizing Mobile Broadband with MongoDB

Importing the Speed Test Files

Page 18: Visualizing Mobile Broadband with MongoDB

Aggregation

Validated Mobile tests are stored in a collection for aggregation.

$geoIntersects operator is used to determine the geographies that each test is contained by.

Page 19: Visualizing Mobile Broadband with MongoDB

Aggregation

Page 20: Visualizing Mobile Broadband with MongoDB

Now, Let's Make a Map!

Page 21: Visualizing Mobile Broadband with MongoDB

Actually, let's make 36 maps!

Page 22: Visualizing Mobile Broadband with MongoDB

Developing the API/api/mmba/carrier/verizon.json?lat=34&lon=-118.2

json, jsonp, geojson, and xml are supported if (validLatLon(lat, lon)) {

console.log('lon ' + lon); console.log('lat ' + lat);

lat = Number(lat); lon = Number(lon);

geo_find = {geometry: { $geoIntersects: { $geometry: { type: 'Point', coordinates: [lon, lat] }}}}; }

Page 23: Visualizing Mobile Broadband with MongoDB

Developing the UI

Some inspiration from the Yahoo Weather App

Page 24: Visualizing Mobile Broadband with MongoDB

Iterate the Design

Page 25: Visualizing Mobile Broadband with MongoDB

Finish with something everyone is proud of

Page 26: Visualizing Mobile Broadband with MongoDB

Putting it all togetherPlanned Release in August 2014: Web application, API, Developer Resources, Data Download, and all source code released on GitHub.

Page 27: Visualizing Mobile Broadband with MongoDB

Next Steps:Deploy the web applicationStart the next cycle.Get more data!

Page 28: Visualizing Mobile Broadband with MongoDB

With Heartfelt Thanks

—Computech Team—FCC staffers —Sam and team at SamKnows, UK —MongoDB Support

Page 29: Visualizing Mobile Broadband with MongoDB

Download the FCC Speed Test App from the Google Play or iTunes App Store and help us put the CROWD in crowdsouce.www.fcc.gov/measuring-broadband-america