Big Data

What is Big data?

Big Data refers to the massive amounts of data that collect over time that are difficult to analyze and handle using common database management tools.

The data are analyzed for marketing trends in business as well as in the fields of manufacturing, medicine and science.

The types of data include business transactions, e-mail messages, photos, surveillance videos, activity logs and unstructured text from blogs and social media, as well as the huge amounts of data that can be collected from sensors of all varieties

Who's Generating Big Data?

Social media and networks(all of us are generating data)

Scientific instruments(collecting all sorts of data)

Mobile devices (tracking all objects all the time)

Sensor technology and networks(measuring all kinds of data)

Most analysts and practitioners currently refer to data sets from 30-50 terabytes(1000 gigabytes per terabyte) to multiple petabytes (1000 terabytes per petabyte) as big data.

Big data: 3V's

Volume:The massive scale and growth of unstructured data outstrips traditional storage and analytical solutions

Velocity:Data is generated in real time, with demands for usable information to be served up immediately

Variety: Data is getting generated in the form of relational data, text data, semi structured data ,Graph data etc.

Examples of Big Data Projects

Consumer product companies and retail organizations are monitoring social media like Facebook and Twitter to get an unprecedented view into customer behavior, preferences, and product perception.

Manufacturers are monitoring minute vibration data from their equipment, which changes slightly as it wears down, to predict the optimal time to replace or maintain. Replacing it too soon wastes money; replacing it too late triggers an expensive work stoppage

Advertising and marketing agencies are tracking social media to understand responsiveness to campaigns, promotions, and other advertising mediums.

- - one of largest Destinations on the web

80% of the U.S.Internet population uses Yahoo!

Global network of content,commerce ,media ,search and access products.

100+ properties including mail ,TV, news ,shopping ,finance,autos ,travels,games ,movies, healths ,etc.

25+ terabytes of data collected each day Representing 1000's of cataloged consumer

behaviours

Yahoo!Big Data-A league of its own

Grand challenge problems of data processing

Travel,Credit card processing ,Stock exchange ,Retail,Internet

Y!Data challenge exceeds others by 2 orders of magnitude

Behavioral Targeting(BT)

Yahoo!User DNA

On a per consumer basis: maintain a behavioral/interests profile andprofitability (user value and LTV) metrics

Row 1 Row 2 Row 3 Row 40

2

4

6

8

10

12

Column 1

Column 2

Column 3

Thank you