Big data deep learning: applications and challenges
-
Upload
fazail-amin -
Category
Data & Analytics
-
view
112 -
download
0
Transcript of Big data deep learning: applications and challenges
![Page 1: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/1.jpg)
Big data Deep learning applications and challenges
![Page 2: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/2.jpg)
Contents● Introduction to big data● Big data challenges● Big data analytics● Applications of big data analytics● Deep learning● Application of deep learning in big
data analytics● Challenges in big data deep learning
![Page 3: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/3.jpg)
Introduction to big data
![Page 4: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/4.jpg)
Introduction
What is big data ?
Big data is loosely defined term
Data is said to be big data when:Volume is very large
Velocity is very high
Variety is broad
As per above definition question arises how do quantify them ?
What should be the limit of terms to make the data big data ?
![Page 5: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/5.jpg)
Need to identify big data
Early detection of big data is necessary To enable proper provisioning of storage and retrieval
To deploy technology for data analytics
For acquisition of requisite expertise for data management and analytics
![Page 6: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/6.jpg)
Big data challenges:
Data analysis is a big challenge due to variety of data which includes structured, semi-structured and unstructured data
Storage of such a large volume of data
Sharing and transfer of data is also a problem when volume and velocity of data generation is that high
Privacy and security of data is serious threat for big data
![Page 7: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/7.jpg)
Big data analytics
![Page 8: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/8.jpg)
Big data analytics:
Big data provides a great opportunity to analyse and extract new information from the data
Due to big data technologies we are able to handle such large volumes of ever increasing variety of data
Data which were considered to be worthless are now being used to exract information
Big data can large datasets to aid machine learning algorithms
![Page 9: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/9.jpg)
Applications of big data analytics
Sensor data analysis
Trend analysis
Management information systems
Cyber physical systems
Network intrusion detection systems
![Page 10: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/10.jpg)
Big data deep learning
![Page 11: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/11.jpg)
Deep learningDeep learning algorithms extract high level complex abstractions
They develop layered hierarchical architecture of learning and data representation
Higher level abstractions are defined in terms of lower-level abstractions
When learning from large amount of unsupervised data in a greedy fashion deep learning algorithms perform quite well
![Page 12: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/12.jpg)
Applications of deep learning in big data:
Extraction from Unlabeled and unsupervised data: Deep learning algorithms can be applied to large amount of unlabeled and unsupervised data for extracting meaningful representations and patterns
Extraction of non-local relationships and patterns:One of the attractive feature of deep learning algorithms is that they can extract non-local relationships and patterns in the data
Context extraction, relational and semantic knowledge extraction
Transfer learning
![Page 13: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/13.jpg)
Practical applications of deep learning in big data analyticsSemantic indexing:
Semantic indexing for search engines: It presents the data in more efficient manner and makes it useful as a source for knowledge discovery and comprehension apart from increasing speed and efficiency
Example: semantic indexing can be used in search engines to make them work fast and more efficiently
In this scheme instead of using raw data for indexing, deep learning can be used to generate high-level abstract data representations for indexing
This semantic indexing works because it allow the data with relatively similar representations to be stored closer to one another in memory aiding in efficient information retrieval
![Page 14: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/14.jpg)
Discriminative tasks and semantic tagging:
Deep learning can be used to extract complicated nonlinear features from the raw data and then use simple linear model to perform discriminative tasks using extracted features as input
MAVIS: Microsoft research audio video indexing system is an example that uses deep learning (ANN) based speech recognition to enable searching of audio and video files with speech
![Page 15: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/15.jpg)
Transfer learning and domain adaptation:Transfer learning: Methods to transfer knowledge/features learned in one or more source
tasks and use it to improve learning in a other related target task
Deep learning can utilize high velocity and variety of big data for transfer learning and domain adaptation
Here training and test data are sampled from different distributions
Deep learning is capable of identifying shared factors present in the input by using transfer learning and domain adaptation
![Page 16: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/16.jpg)
Deep learning challenges in big data analytics
![Page 17: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/17.jpg)
challenges:
There are some problems associated with application of deep learning for big data analytics like:
Learning with fast moving and streaming data
High dimensionality of data
Scalability of the the model
Distributed computing of data
![Page 18: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/18.jpg)
Future research challenges :
Adapting deep learning algorithms to address the analytics of big data
Making learning algorithms work efficiently while focussing on the scalability of the model
Finding criteria for extracting good data representations and domain adaptation
Improving the learning algorithms for distributed computing
![Page 19: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/19.jpg)
References1.Maryam Najafabadi et. el., “deep learning applications and
challenges in big data analytics ”, Journal of Big Data, SpringerOpen Journal 2015.
2.Xue-wen Chen. “big data deep learning: challenges and perspectives”, IEEE, 2014.
3.Huihong He,”using object-oriented big data analytics to reveal server performance dead zone”, IEEE Annual comp. Soft. and applications conf., 2016.
4.Suthaharan Shan, ”big data classification: problems and challenges in network intrusion detection with machine learning”ACM, 2014.
5.Jose Lluis Berral_Garcia, “a quick view on current techniques and machine learning algorithms for big data analytics”, IEEE ICTON, 2016.
![Page 20: Big data deep learning: applications and challenges](https://reader036.fdocuments.net/reader036/viewer/2022062400/58808b2a1a28ab35718b69a3/html5/thumbnails/20.jpg)
Thank you