LogiNext Big Data Workshop NASSCOM

Copyright © LogiNext Solu3ons Inc. All Rights Reserved. Visit www.loginextsolu3ons.com/termsofuse for more details.

1 What?

2 Why?

3 Where?

4 How?

AGENDA

1


WHAT?


How Much Data You Consume & Produce Everyday?


How Much Data You Consume & Produce Everyday?

1 MB

5 MB 0.5 MB

0.5 MB

0.5 MB

1 MB


Big Data Facts

200 Million Emails 1.8 Million FB Likes

Every Minute…

5

300K Tweets 48 hours YouTube Videos


The Mindset Th4Vsindset


WHY?


The Value

COLLECT

STORE

ANALYZE

VISUALIZE


Be daring, be different, be

impractical, be anything that

will assert integrity of

purpose and imaginative

vision against the play-it-

safers, the creatures of the

commonplace, the slaves of

the ordinary.

Data Around Us…









the ordinary.

Make Things Smarter…

•  Critical Alerts

•  Technical Problems

•  Service Alerts

•  Headlight On Warning

•  Location Information

•  Towing Alerts

•  Crash Alerts

•  Over Speeding Alerts

•  Fuel Economy









the ordinary.

Analytics


Visualization

https://housing.com/dsl/heatmaps/mumbai/rent


14

What is the Maximum Speed?

Day Speed (kmph) Day Speed (kmph) Day Speed (kmph)

1 30 11 60 21 90

2 117 12 115 22 60

3 45 13 35 23 63

4 90 14 121 24 68

5 60 15 40 25 79

6 63 16 0 26 108

7 68 17 0 27 40

8 79 18 30 28 123

9 108 19 42 29 70

10 110 20 45 30 83


15

How Many Days the Speed was Above 100?


1 30 11 60 21 90

2 117 12 115 22 60

3 45 13 35 23 63

4 90 14 121 24 68

5 60 15 40 25 79

6 63 16 0 26 108

7 68 17 0 27 40

8 79 18 30 28 123

9 108 19 42 29 70

10 110 20 45 30 83


16

How Many Days Did You Spend at Home?


1 30 11 60 21 90

2 117 12 115 22 60

3 45 13 35 23 63

4 90 14 121 24 68

5 60 15 40 25 79

6 63 16 0 26 108

7 68 17 0 27 40

8 79 18 30 28 123

9 108 19 42 29 70

10 110 20 45 30 83


3 Seconds...

0

20

40

60

80

100

120

140

160

180

200

220

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29


Cars

Sports

WHERE?

Manufacturing

Politics

Education

E-commerce

Health Care

Logistics

Homes

18

Retail


19

What we see!!

Lot of Data

?

What it is!!

Complex Chain

Big Data & Logistics


20

Saves 60 Million USD


https://www.youtube.com/watch?v=ZHqgpVsUfhI


21



22

Predictive & Proactive

•  Know Where The Shipments Are

•  Know How to Deliver Better & Cheaper

•  Make It Easier To Digest For Your Field-Team


Case Study: Last Mile Delivery

Real-time Increased Visibility & Customer Satisfaction

Reduction in Delivery Times

15 %

Increase in Resource Utilization 7%

Reduction in Delays & Errors 12 %

100,000+

5000+

250,000+ KMS

23


BREAK


HOW?


BIG DATA LANDSCAPE

Source: hEp://www.forbes.com/sites/davefeinleib/2012/07/24/big-‐data-‐trends/


DATA ARCHITECTURE

Source: hEp://www.tricoresolu3ons.com/resources/blog/best-‐prac3ces-‐in-‐bi-‐think-‐big-‐start-‐small-‐ensuring-‐a-‐successful-‐bi-‐big-‐da


DATA ARCHITECTURE


STORAGE MongoDB


Data Sources

“In God We Trust, All Others Bring Data” – W.E. Deming

Mobile Apps for Delivery Boys

Flight Data

Matchbox Sized Smart Tags

External Sensors & ERP Data

Train Data

Traffic & Weather Data

30


•  Easy & Dynamic Scalability

•  Flexible Schema

•  Semi-structured & Unstructured Data

•  Impedance Mismatch Between Application and Database

•  Relationship Between Data Not Always Required in Database

•  Better Performance

•  Low Cost

•  Auto-Sharding

•  Distributed Query Support

•  Integrated Caching

Why NoSQL?

The Demands of Your Database Are Changing

31


NoSQL - Scalability

32 Source: hEp://www.couchbase.com/nosql-‐resources/what-‐is-‐no-‐sql


NoSQL - Scalability

33 Source: hEp://www.couchbase.com/nosql-‐resources/what-‐is-‐no-‐sql


NoSQL – Schemaless

34

Storage Is Not A Concern Any More

Source: hEp://www.couchbase.com/nosql-‐resources/what-‐is-‐no-‐sql


NoSQL Types

KEY- VALUE

WE OFFER THE BEST THERE IS 35

GRAPH COLUMN DOCUMENT

OrientDB


Why MongoDB?

36

•  Document Storage

•  BSON - Binary JSON

•  Easy Import & Querying (mongoimport -f filename.json)

•  Friendly with JS

•  Super Easy Installation

•  Querying

•  Rich JS Based Query Syntax

•  Allows Deep Nested Queries

•  Very Powerful For Analysis

“MongoDB (from "humongous") is an open-source document database, and the leading

NoSQL database” – mongodb.org


MongoDB – Geospatial Features

37

•  GeoJSON Objects

•  Point

•  LineString

•  Polygon

•  MultiPoint

•  MultiLineString

•  MultiPolygon

•  GeometryCollection

•  Legacy Coordinate Pairs

“MongoDB offers a number of indexes and query mechanisms to handle

geospatial information”


MongoDB – Query Operations

38

•  Inclusion - $geoWithin

{ <location field>: { $geoWithin: { $geometry: { type: <"Polygon”> , coordinates: [ <coordinates> ] } } } }



39

•  Intersection- $geoIntersects

{ <location field>: { $geoIntersects: { $geometry: { type: "Polygon” , coordinates: [ <coordinates> ] } } } }



40

•  Proximity- $near

{ $near: { $geometry: { type: "Point" , coordinates: [ <longitude> , <latitude> ] }, $maxDistance: <distance in meters>, $minDistance: <distance in meters> } }


MongoDB – Talent

41 Source: hEp://blogs.the451group.com/informa3on_management/2014/10/01/nosql-‐linkedin-‐skills-‐index-‐september-‐2014/


42

LogiNext - All In One


Case Study: Avg. E-Commerce Company

43

100,000+ Shipments Per Day

10,000+ PIN Codes Served

300+ Hubs Across India

Shipments Get Delayed

Hands Change For Each Delivery

40%

8%

6 hrs

Of The Time Goes Without Visibility Into Location of Package

7

Saved Per Month

Idle Time for Packages

7 Lacs INR

20%

Business

Challenges Results

More Accuracy in Tracking SLAs


Share Your Use Case


ONLINE

MySQL

MongoDB

OFFLINE

Hadoop

EDW

Big Data Management

45

Real-time Low-latency High availability

Long-running High-latency Availability Low Priority


46

Big Data Management


VISUALIZATION TECHNIQUES

Tag Cloud

47

Heat Map

Spatial Information Flow Network Graphs


VISUALIZATION TOOLS & TECHS

48

•  Tools

•  Tableau

•  SAS

•  Maps

•  Google Maps

•  OpenStreetMaps

•  CartoDB

•  MapMyIndia

•  Charts

•  D3.js

•  HighCharts

•  GoogleCharts


CHALLENGES

Insufficient Budget

49

Adoption

Technology

Technical Skills


OPPORTUNITY

DATA iS GROWING


Please don’t hesitate

Questions


Thank You

for your precious time and attention


THAT’S ME!!

53

LogiNext Big Data Workshop NASSCOM

Data & Analytics

Transcript of LogiNext Big Data Workshop NASSCOM