Big Data Testing for a Learning Management System · Test data transformation - data mismatch,...

Case Study: Big Data Testing for a Learning Management System 33% increase in the performance of a Learning Management System for a Leading Education Provider BUSINESS REQUIREMENTS DATA VALIDATION WORKFLOW Data Integrity validation on Data ingestion layer & Data processing layer Automated data validation with Zero data loss validation across different layers Query performance monitoring, Source - target data mapping 4V data validation (variety, volume, velocity and veracity) Test Analytically-proven assessments, multi-media remedial content and video based real-life simulations Cross platform support for streaming online, interactive online content and printed materials Test data transformation - data mismatch, miscalculation and/or loss of data Cloud Services Amazon AWS S3, Qubole Server Implementation Scala/Java, Sqoop, Oozie Database Hive, Oracle Data Mart Test Tools Java, JDBC, MRUnit Data Visualization Tableau OUR SOLUTIONS TOOLS /TECHNOLOGIES BUSINESS IMPACT Quicker response time was achieved by fine tuning and redesigning DB query and adding nodes to cluster 33% increase in the performance upon partitioning and categorizing data About Happiest Minds Technologies: Happiest Minds, the Mindful IT Company, applies agile methodologies to enable digital transformation for enterprises and technology providers by delivering seamless customer experience, business efficiency and actionable insights. We leverage a spectrum of disruptive technologies such as: Big Data Analytics, AI & Cognitive Computing, Internet of Things, Cloud, Security, SDN-NFV, RPA, Blockchain, etc. Positioned as “Born Digital . Born Agile”, our capabilities spans across product engineering, digital business solutions, infrastructure management and security services. We deliver these services across industry sectors such as retail, consumer packaged goods, edutech, e-commerce, banking, insurance, hi-tech, engineering R&D, manufacturing, automotive and travel/transportation/hospitality. Headquartered in Bangalore, India; Happiest Minds has operations in USA, UK, The Netherlands, Australia and Middle East. To know more about our offerings. Please write to us at [email protected] User Application events stored in Cassandra DB Sqoop Extract Perform Transformations Validate fallover & recoverability scenarios - Validate Workflows Load DataMart for Scientists/Analysts Oracle Data Marts Reports Data Availability, and Data Formatting Validate Source vs Targets Check for - Data Loss, Data Integrity and Data Mapping Data Archived in S3 (JSON) Qubole

Upload
others
Category

Documents
view
1
download
0

Embed Size (px):

Transcript of Big Data Testing for a Learning Management System · Test data transformation - data mismatch,...

Page 1: Big Data Testing for a Learning Management System · Test data transformation - data mismatch, miscalculation and/or loss of data Cloud Services Amazon AWS S3, Qubole Server Implementation

Case Study: Big Data Testing for a Learning Management System

33% increase in the performance ofa Learning Management System for a Leading Education Provider

BUSINESSREQUIREMENTS

DATA VALIDATION WORKFLOW

Data Integrity validation on Data ingestion layer & Data processing layer

Automated data validation with Zero data loss validation across different layers

Query performance monitoring,Source - target data mapping

4V data validation (variety, volume, velocity and veracity)

Test Analytically-proven assessments, multi-media remedial content and video based real-life simulations

Cross platform support for streaming online, interactive online content and printed materials

Test data transformation - data mismatch, miscalculation and/or loss of data

Cloud ServicesAmazon AWS S3, Qubole

Server ImplementationScala/Java, Sqoop, Oozie

DatabaseHive, Oracle Data Mart

Test Tools Java, JDBC, MRUnit

Data VisualizationTableau

OURSOLUTIONS

TOOLS/TECHNOLOGIES

BUSINESSIMPACT

Quicker response time was achieved by fine tuning and redesigning DB query and adding nodes to cluster

33% increase in the performance uponpartitioning and categorizing data

About Happiest Minds Technologies:

Happiest Minds, the Mindful IT Company, applies agile methodologies to enable digital transformation for enterprises and technology providers by delivering seamless customer experience, business efficiency and actionable insights. We leverage a spectrum of disruptive technologies such as: Big Data Analytics, AI & Cognitive Computing, Internet of Things, Cloud, Security, SDN-NFV, RPA, Blockchain, etc. Positioned as “Born Digital . Born Agile”, our capabilities spans across product engineering, digital business solutions, infrastructure management and security services. We deliver these services across industry sectors such as retail, consumer packaged goods, edutech, e-commerce, banking, insurance, hi-tech, engineering R&D, manufacturing, automotive and travel/transportation/hospitality.Headquartered in Bangalore, India; Happiest Minds has operations in USA, UK, The Netherlands, Australia and Middle East.

To know more about our offerings. Please write to us at [email protected]

User Applicationevents stored in

Cassandra DB

Sqoop Extract

PerformTransformations

Validate fallover &recoverability scenarios- Validate Workflows

Load DataMart forScientists/Analysts

OracleData Marts

Reports

Data Availability,and DataFormatting

Validate Sourcevs Targets

Check for - DataLoss, Data Integrityand Data Mapping