Data Analytics: - An Introduction

Ashish Bhasin (ashbhasin@gmail.com )

Introduction

The explosion of data across enterprises has forced consultants to think beyond Gigs of data and work on their Data Management strategy. Both structure and unstructured Data has been growing for enterprise and enterprises are leaving no stone unturned to analyze the consumer behavior. Companies key ask from technology companies is to analyze the trend from the terabytes of data so that they can fine tune their offerings to compete in the market space by targeting key consumer want to enlarge the market share of the product.

Enterprises are expecting real time analysis of the data and application providing reports which could enable companies taking informed decisions. However with the increase in volumes of data, organizations are facing problem correlating these data (Fig 1):-

Length and breadth: - Number of records/Tables in database is increasing as companies trying to capture real time data with number of observed parameters increasing.

Low Latency for real time data Multiple data sources: - Source of data collection has increased, could be feedback from

internet, collection from grounds. Data Quality: - Reliable data source is another challenge which organization is facing.

Major Transition in database since last decade, has been shifting of data from structure data to unstructured data and this has lead to the increase in complexity for managing databases. Companies have reached closer to consumer with Social networking and are expecting to analyze consumer behavior/feedback almost on the realtime. Companies are aware that neglecting the consumer sentiments could lead them out of the market.

Figure 1 :- Data Trend

Backbone of Data Analytics: - 3I’s of Analytics

For any organization, companies need to focus on 3 I’s to ensure that they could for having are always intact and adapting as per industry requirements:-

Infrastructure: - Enterprise needs to ensure data collection techniques are as per reviewed and fine tune periodically. Enterprises have to collect data from various sources which could be through interaction with consumer. Putting their like/suggestion and dislike of the product. Various agencies, conduct survey to collect the data from consumer.

Interconnection :- Data once collected has to run through various batch jobs to interlink the data and various algorithm specific to industry produce a context

Intelligence: - Various reports need to be provisioned for stakeholder to show trends of various scenario.

Figure 2 :- 3i’s of Analytics

InfrastructureCaptures data about what is going on in the world. Who is where etc.

InterconnectionLevel of Interconnection between people, public safety and other government agencies, and the private sector.

IntelligenceUse Predictive modeling to Analyze the Data Ensure Preventive action to reduce probability.

High Level Structure of Analytics

Fig3 :- Trend of Analytics

Figure 3 shows process flow for an enterprise having Analytics infrastructure, where Data collected from multiple sources are collated at a single data warehouse and various batch job are done to represent meaningful output to the stakeholders. In this section, we will explore various components of the ecosystem:-

Data Sources: - Enterprises collates data from various systems (external/internal i.e. Internet/ survey/ customer feedback/selling trend etc). This data is collated at a single data warehouse to ensure a holistic view is presented.

ETL :- This components convert data into a readable format before putting in Data warehouse. This component is usually customized for each industry/company.

Datawarehouse: - This is a heart of Analytics system where all data is collected. Sizing of database is done depending upon requirement from an enterprise, past data might be included.

BI Service components: - This is a Brain of all analytics system, responsible for running algorithm on data to make a correlation/trend in the data and could provide a predict behavior of the customer.

Reports:- Depending upon position/requirement of company customized reports could be produced so that marketing organization can predict a churn in certain section and initiate a dedicated campaign to prevent the same.

Ashish Bhasin (ashbhasin@gmail.com)