Page History

...

• Kafka Streams - the built-in stream processing library of the Apache Kafka project
  
  • Documentation in Apache Kafka
  • Documentation in Confluent Platform
  • Kafka Streams code examples in Apache Kafka
  • Kafka Streams code examples provided by Confluent
• Storm - A stream-processing framework.
• Samza - A YARN-based stream processing framework.
• Storm Spout - Consume messages from Kafka and emit as Storm tuples
• Kafka-Storm - Kafka 0.8, Storm 0.9, Avro integration
• SparkStreaming - Kafka receiver supports Kafka 0.8 and above
• Flink - Apache Flink has an integration with Kafka
• IBM Streams - A stream processing framework with Kafka source and sink to consume and produce Kafka messages 

Hadoop Integration

• Kafka Connect sink Confluent HDFS Connector - A sink connector for the Kafka 's connector framework.Connect framework for writing data from Kafka to Hadoop HDFS
• Camus - LinkedIn's Kafka=>HDFS pipeline. This one is used for all data at LinkedIn, and works great.
• Kafka Hadoop Loader A different take on Hadoop loading functionality from what is included in the main distribution.
• Flume - Contains Kafka Source source (consumer) and Sink sink (producer)
• KaBoom - A high-performance HDFS data loader

Database Integration

• Confluent JDBC Connector - A source connector for the Kafka Connect framework for writing data from RDBMS (e.g. MySQL) to Kafka

Search and Query

• ElasticSearch - This project, Kafka Standalone Consumer will read the messages from Kafka, processes and index them in ElasticSearch. There are also several Kafka Connect connectors for ElasticSeach.
• Presto - The Presto Kafka connector allows you to query Kafka in SQL using Presto.
• Hive - Hive SerDe that allows querying Kafka (Avro only for now) using Hive SQL

...