Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented...

14
Real World Big Data Jorge Castro, Kevin Monroe

Transcript of Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented...

Page 1: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Real World Big DataJorge Castro, Kevin Monroe

Page 2: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Juju & Big Data

● Modeling language for service oriented environments

○ Deploy, relate, scale○ Reliable○ Repeatable○ Observable

Page 3: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Plugging in to Hadoop

juju quickstart apache-core-batch-processing

Page 4: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Add Apache Flume

juju quickstart u/bigdata-dev/apache-ingestion-flume

Page 5: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Add Apache Kafka

juju quickstart u/bigdata-dev/apache-flume-ingestion-kafka

Page 6: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Add Apache Pig

juju quickstart apache-analytics-pig

Page 7: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

juju quickstart apache-analytics-sql

Add Apache Hive

Page 8: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

juju quickstart apache-hadoop-spark-notebook

Add Spark and iPy Notebook

Page 9: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

juju quickstart apache-hadoop-spark-zeppelin

Add Apache Zeppelin

Page 10: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Build Your Solution

Page 11: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Syslog Analytics

juju quickstart realtime-syslog-analytics

Page 13: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Resources

● Mailing Lists○ Juju Big Data: https://lists.ubuntu.com/mailman/listinfo/bigdata○ Juju: https://lists.ubuntu.com/mailman/listinfo/juju

● IRC (Freenode)○ #juju

● Web○ bigdata.juju.solutions○ developer.juju.solutions

■ Free AWS creds!!○ jujucharms.com/docs

Page 14: Real World Big Data - SCALE · PDF fileJuju & Big Data Modeling language for service oriented environments ... plugin flume-hdfs zookeeper flume-kafka kafka yarn-master compute-slave

Thanks!