Apache kafka

Apache KafkaYour step 1 to Big Data

Who am I
Saurabh Minni also on the web as @the100rabh
Yet another developer in Bangalore
I just love tinkering with different technologies
Currently working as Technical Architect at Near
Been part of planning for Barcamp Bangalore since 2007
Author of Apache Kafka Cookbook - https://www.
packtpub.com/big-data-and-business-intelligence/apache-
kafka-cookbook
https://twitter.com/the100rabhhttps://www.packtpub.com/big-data-and-business-intelligence/apache-kafka-cookbookhttps://www.packtpub.com/big-data-and-business-intelligence/apache-kafka-cookbookhttps://www.packtpub.com/big-data-and-business-intelligence/apache-kafka-cookbookhttps://www.packtpub.com/big-data-and-business-intelligence/apache-kafka-cookbook

Features of Kafka
Fast
Scalable
Durable
Distributed by Design

Architecture of an era
Node Processor

Architecture of an scaling era - II
Front End Node
Hadoop
Front End Node
Front End Node

Things got a little more complicated
Front End Node
Hadoop
Front End Node
Front End Node
Realtime Processing
System

Apache Kafka to the rescue
Front End Node
Apache Kafka
Front End Node
Front End Node
Hadoop
Apache Storm

We need more
Front End Node
Apache Kafka Cluster
Front End Node
Front End Node
Hadoop
Apache Storm
Apache Kafka Node
Apache Kafka Node
Apache Kafka Node
Apache Kafka Node

Some more details
Kafka Cluster
Apache Kafka Node
Apache Kafka Node
Apache Kafka Node
Apache Kafka Node
Producer Kafka Topic
Producer Kafka Topic
Producer
Consumer (Hadoop)
Consumer (Apache Storm)

What is Apache Kafka
Apache Kafka is publish-subscribe
messaging rethought as a distributed
commit log.

Monitoring Kafka

Which frameworks think Kafka is good

Which companies use Kafka
Probably a better question would be
which significantly large companies
dont use Kafka

Where Apache Kafka gives trouble?
Requires ZooKeeper
Complexity in making a consumer
A lot of manual processes
Management tools leave a lot to be desired

Apache kafka

Software

Transcript of Apache kafka

Publish-subscribe Message Framework with Apache Kafka and ... · PDF file Introduction of Kafka Kafka elementary concepts Kafka architecture ... Message Framework with Apache Kafka

Apache Kafka Overview Kafka Introduction Apache Kafka is a high performance, highly available, and redundant streaming message platform. Kafka functions much like a publish/subscribe

Integrating Apache Hive with Kafka, Spark, and BI · PDF file 2020-05-10 · Data Access Apache Hive-Kafka integration Related Information Apache Kafka Documentation Perform ETL by

Apache Kafka Security

Kafka Streams: The Stream Processing Engine of Apache Kafka

Authorization in Apache Kafka - Seattle Kafka Meetup - Ashish Singh

Apache: Big Data 2015 Kafka Architecture The Best of Apache · PDF fileThe Best of Apache Kafka Architecture Ranganathan Balashanmugam @ran_than Apache: Big Data 2015

Apache Kafka - · PDF fileOverview What is Apache Kafka? Data pipelines Architecture How does Apache Kafka work? Brokers Producers Consumers Topics

Spring for Apache Kafka ... Spring for Apache Kafka 2.1.5.RELEASE Gary Russell , Artem Bilan , Biju Kunjummen

Apache kafka

Slides - Apache Kafka® Architecture & Fundamentals Explained · PDF file for Apache Kafka (aligns to Confluent Developer Skills for Building Apache Kafka course) Confluent Certified

Apache kafka intro_20150313_springloops

Kafka Tutorial - Introduction to Apache Kafka (Part 1)

Using Apache Spark, Apache Kafka and Apache Cassandra ... USING APACHE SPARK, APACHE KAFKA AND APACHE CASSANDRA TO POWER INTELLIGENT APPLICATIONS | 02 Apache Cassandra is well known

Making Apache Kafka Elastic with Apache Mesos

Introduction to Apache Kafka

Apache Kafka Lesson Learned

Stream Processing using Apache Spark and Apache Kafka

Apache Kafka - Free Friday

· PDF fileApache Kafka Introduction to Apache Kafka Apache Kafka Architecture explanation Practical Examples on Apache Kafka SCALA, PYTHON, SPARK Course Content