PowerPoint Presentation
Big Data && Apache Spark
Hakkmda
stanbul nv . Bilgisayar Mh
9+ Java , Java EE
3+ Hadoop,Spark,Pig,Hive,Oozie
Big Data Developer - Comodo
Blogger/Trainer - buyukveri.co
erik
Byk Veri(Big Data) Nedir ?
Hadoop Nedir ve Temel Bileenleri (HDFS , MapReduce)?
Kullanm Senaryolar
Apache Spark
Apache Spark Bileenleri
Apache Spark Mimarisi
RDD,DataFrame,DataSet
Spark Streaming
Mlib
Byk Veri Nedir?
Big data(byk veri) genel olarak yksek hacimlerdeki veriler iin kullanlan bir terimdir.
Volume(Veri Hacmi)
Bu zellik verinin yksek hacimli olduunu belirtir
Velocity(Veri hz)
Veriler o kadar hzl retilmektedir ki , bir dakika ierisinde 200+ milyon e-mail , 4+ milyon Facebook like , 1+ milyon Instagram like ilemi yaplmaktadr
Variety(Veri eitlilii)
Verilerin belirli bir yaps yoktur, genellikle deikendir.Resimler, ses dosyalar, text dosyalar
Hadoop Nedir ?
Hadoop , byk veri kmeleri ile birden fazla makinada paralel olarak ilem yapmamz salayan Java ile yazlm ak kaynak kodlu ktphanedir
Hadoop Hdfs Nedir ?
Hadoop ierisinde byk verileri sakladmz bileene HDFS (Hadoop Distributed File System) denir
Hadoop MapReduce Nedir?
Hadoop ierisinde byk verileri paralel olarak ileyebileceimiz bileene MapReduce denir .
Kullanm Senaryolar
rnek Analizler
Son bir saat ierisinde 50 den fazla reklama tklayan kullanclar
Referer_url bilgisinden arama yaplan anahtar kelimelerin bulunmas
Tavsiye motoru (Mzik siteleri , sinema siteleri , alveri siteleri)
Network zerinde anormal durumlar (fraud , anomaly detection)
Reklam gsterimleri (Ad-network)
Apache Spark
Apache Spark byk verileri datk makinalarda ilemek iin kullanlan bir ktphanedir
Apache Spark
2009 ylnda UC Berkeley ' de gelitiriliyor
Hadoop ile karlatrldnda 10(x) disk ve 100(x) memory hzl
Java, Scala, Python, R
u an Apache Software Foundation altnda gelitiriliyor
2014 ylnda TLP(Top Level Project) olarak duyuruluyor
Apache Spark Bileenleri
Apache Spark Mimarisi
RDD, DataFrame, DataSet
Verileri cluster zerinde saklayarak paralel olarak ilemler yapmamz salar
RDD
Datk(Distributed) JVM nesneleridir
map,reduce,count,groupBy,filter,join,saveAsTextFile .. gibi metodlar salar
DataFrame
Spark 1.3 ile gelmitir
RDD ye gre performans daha iyidir . SQL benzeri sorgular altrlabilir.
DataSet
Spark 1.6 ile gelmitir
Type-safe salar ,hatalar compile time da gsterir
Memory zerinde daha az yer kaplar
RDD ' ye gre hzldr
Encoder yaps deimitir
Spark Streaming
Gelen verileri anlk olarak(real time) analiz etmemizi salar
Toplad verileri parametrik olarak window lara ayrr (window1,window2...)
MLib
leklenebilir machine learning ktphaneleri mevcuttur
Java, Scala, Python ve R ile gelitirilebilir
Logistic regression
Gelen e-mailler ierisinde spam olanlar belirlenebilir
Clustering
K-means ile network verileri zerinde anormal ataklarn(anomaly detection) bulunmas
Recomendation
ALS ile bireylerin davranlarnn birbirlerine benzetilerek film,mzik.. nerisi
K-means
ALS
Teekkrler
Balk metni biimini dzenlemek iin tklaynClick to edit Master title style
11/25/16
Edit Master text styles
Second levelThird levelFourth levelFifth level
Click to edit Master title style
Click to edit Master subtitle style
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Edit Master text styles
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
11/25/16
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
Second levelThird levelFourth levelFifth level
11/25/16
Click to edit Master title style
Edit Master text styles
Second levelThird levelFourth levelFifth level
11/25/16
Balk metni biimini dzenlemek iin tklaynClick to edit Master title style
Anahat metni biimini dzenlemek iin tklayn
kinci Anahat Dzeyi
nc Anahat Dzeyi
Drdnc Anahat Dzeyi
Beinci Anahat DzeyiAltnc Anahat DzeyiYedinci Anahat DzeyiSekizinci Anahat Dzeyi
Dokuzuncu Anahat DzeyiEdit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Click to edit Master subtitle style
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Edit Master text styles
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
11/25/16
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Click to edit Master title style
Edit Master text styles
Second level
Third level
Fourth level
Fifth level
11/25/16
Top Related