Real Time Analytics via Spark & Scala | Spark & Scala Fundamentals | Spark & Scala...

download Real Time Analytics via Spark & Scala | Spark & Scala Fundamentals | Spark & Scala Architecture

of 22

Embed Size (px)

Transcript of Real Time Analytics via Spark & Scala | Spark & Scala Fundamentals | Spark & Scala...

  1. 1. Slide 1 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Real Time Analytics via Spark & Scala
  2. 2. Slide 2 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Session Objectives Introduction to Big Data and Hadoop Understanding Spark Why Spark? Note on Scala Getting answers to interview questions on Spark Getting your doubts cleared
  3. 3. Slide 3 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Big Data and its Challenges
  4. 4. Slide 4 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Big Data and its Challenges Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications Systems / Enterprises generate huge amount of data from Terabytes to and even Petabytes of information Its very difficult to manage such huge data Get Started with Spark and Scala
  5. 5. Slide 5 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Who Generates Big Data? Have you ever wondered how Google, Facebook or LinkedIn manages to store and utilize the huge data? Today, it is becoming a problem for all of us to manage such BIG DATA. Get Started with Spark and Scala
  6. 6. Slide 6 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Spark - Lightens Fast Cluster Computing Apache Spark is an open source big data processing framework Spark is used at a wide range of organizations to process large datasets 1. Easy to develop 2. Interactive shell 3. Out of box functionality Get Started with Spark and Scala
  7. 7. Slide 7 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Spark? 100x Faster than Speed: Run programs up to 100x faster than Hadoop Map Reduce in memory, or 10x faster on disk. Get Started with Spark and Scala
  8. 8. Slide 8 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Spark? Ease of Use: Allows to write applications quickly in Java, Scala or Python. Get Started with Spark and Scala
  9. 9. Slide 9 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Spark? Generality: Combines SQL, streaming, and complex analytics. Get Started with Spark and Scala
  10. 10. Slide 10 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Spark? Runs Everywhere: Runs on Hadoop, Mesos, standalone, or in the cloud. Get Started with Spark and Scala
  11. 11. Slide 11 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Spark Architecture Spark Architecture includes following three main components: Data Storage API Management Framework API (Scala, Java, Python) Storage (HDFS other formats) Distributed Computing (Stand-alone, Mesos, YARN) Compute Interface Data Management Get Started with Spark and Scala
  12. 12. Slide 12 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Spark Ecosystem Spark Core Engine Shark (SQL) Spark Streaming (streaming) MLLib (Machine Learning) GraphX (Graph Computation) SparkR (R on Spark) BlinkDB (Approximate SQL) Alpha/Pre-alpha Get Started with Spark and Scala
  13. 13. Slide 13 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com The Spark Users Get Started with Spark and Scala
  14. 14. Slide 14 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Scala Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way It integrates features of object-oriented and functional languages Publicly released in January 2004 on the JVM Platform and a few months later on the .NET platform Get Started with Spark and Scala
  15. 15. Slide 15 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Scala? Scala is a type-safe JVM language that incorporates both object oriented and functional programming into an extremely concise, logical, and extraordinarily powerful language Scala makes use of a lot of functional syntactic sugar that has become popular with developers and makes many developers characterize Scala as a more functional language Scala is Statically typed Language Being used heavily for future Big data and developments frameworks like Spark, Akka, Scalding etc. Get Started with Spark and Scala
  16. 16. Slide 16 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Job Trends Spark and Scala Get Started with Spark and Scala
  17. 17. Slide 17 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why SkillSpeed? Course Curriculum from Industry Experts Instructor Led Live Virtual Sessions Lifetime access to Course Content via LMS 100% Placement Assistance 24x7 Support Get Started with Spark and Scala
  18. 18. Slide 18 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Course Topics Module 1 Getting Started / Introduction to Scala Module 2 Scala Essentials and Deep Dive Module 3 Introducing Traits and OOPS in Scala Module 4 Functional Programming in Scala Module 5 Spark and Big Data Module 6 Advanced Spark Concepts Module 7 Understanding RDDs Module 8 Shark, SparkSQL and Project Discussion Get Started with Spark and Scala
  19. 19. Slide 19 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Corporate Partners Get Started with Spark and Scala
  20. 20. Slide 20 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Lines open 24/7 To know more about the course, Please contact: IND +91-90660-20904 USA 1866-607-6547 (Toll Free) Or reach us at sales@skillspeed.com Contact us.. Get Started with Spark and Scala
  21. 21. Slide 21 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Image References Google images credit for google, Facebook and LinkedIn LOGO and Snapshots http://pixshark.com/big-data-comic.htm http://findicons.com/icon/66444/user_group http://www.virtualizor.com/tour https://accounts.it.et.byu.edu/ http://www.clipartsfree.net/tag/server.html http://www.gopixpic.com/16/time-clock-icon-png-download http://blog.smartbear.com/requirements/how-to-interview-users-to-find-out-what-they-really-want/ http://www.lincs.fr/research/areas/big-data/ http://www.counsellingpages.co.uk/ http://langfordsconsultancy.com/langfords-training-support-package/ http://cbsepathshala.blogspot.in/2012/05/physics-class-x-chapter-electricity.html http://mmatycoon.com/tycoontimes/tycoontimesstory.php?SID=1010