Ecosystem

Here is a list of tools we have been told about that integrate with Kafka outside the main distribution. We haven't tried them all, so they may not work!

Clients, of course, are listed separately here.

Hadoop Integration

• Camus - LinkedIn's Kafka=>HDFS pipeline. This one is used for all data at LinkedIn, and works great.
• Kafka Hadoop Loader A different take on Hadoop loading functionality from what is included in the main distribution.

AWS Integration

• Automated AWS deployment
• Kafka->S3 Mirroring

Logging

• klogd - A python syslog publisher
• klogd2 - A java syslog publisher
• Tail2Kafka - A simple log tailing utility
• Fluentd plugin - Integration with Fluentd
• Flume Kafka Plugin - Integration with Flume
• Remote log viewer
• LogStash integration - Integration with LogStash and Fluentd

Metrics

• Mozilla Metrics Service - A Kafka and Protocol Buffers based metrics and logging system
• Ganglia Integration

Packing and Deployment

• RPM packaging
• Debian packaginghttps://github.com/tomdz/kafka-deb-packaging
• Puppet integration
• Dropwizard packaging

Misc.

• Kafka Mirror - An alternative to the built-in mirroring tool
• Ruby Demo App 
• Apache Camel Integration
• Infobright integration