Spark Streaming on AWS -S3‹‚‰Kinesis¸

download Spark Streaming on AWS -S3‹‚‰Kinesis¸

of 28

Embed Size (px)

Transcript of Spark Streaming on AWS -S3‹‚‰Kinesis¸

  • Spark Streaming on AWS

    -S3Kinesis- !

    2015-09-25 JAWS-UG Meguro #2

  • @chibochibo03

    CTO

    Scala

    GitBucket

    https://github.com/takezoe/gitbucket

    https://github.com/takezoe/gitbucket

  • Scala

    Elasticsearch

    AWS

  • Spark

  • SparkES

    ??

  • Spark Streaming

    EC2Standalone Cluster

    Apache Storm()

  • SparkES

    ??

    Spark

  • S

  • S3

    Spark

    1(--total-executor-cores)

  • S3

  • Kinesis

  • Kinesis

    Spark1.4KafkaKinesis

  • mackerel

    StreamingListener