Versions Compared

Old Version 86

changes.mady.by.user guozhang Wang

Saved on

compared with

New Version Current

changes.mady.by.user Maciej Mensfeld

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

https://github.com/edenhill/librdkafka

 


Kafka Version: 0.7.x, 0.8.x, 0.9.x, 0.10.x, 0.11.x
Maintainers: Magnus Edenhill

...

-------------

Native C++ library with protocol support for Metadata, Produce, Fetch, and Offset.


Kafka Version: 0.8.x

Maintainer: David Tompkins
License: Apache 2.0

https://github.com/adobe-research/libkafka 


-------------

C++ Header-only Kafka Client Library using Boost Asio

 


Kafka Version: 0.8.x

Maintainer: Daniel Joos
License: MIT

https://github.com/danieljoos/libkafka-asio

 


-------------

A C++11 asyncronous producer/consumer library for Apache Kafka based on boost asio


Kafka Version: 0.8.x

Maintainer: Svante Karlsson
License: Boost Software License - Version 1.0

https://github.com/bitbouncer/csi-kafka

 


-------------

libasynckafkaclient - C++ based single threaded asynchronous library


Kafka Version: 0.8.x

Maintainer: Vijay Jadhav
License: 2-clause BSD

https://github.com/GSLabDev/libasynckafkaclient 


-------------

https://github.com/quipo/kafka-cpp

Kafka Version: 0.7.x

 


-------------

Python

-------------

...

https://github.com/confluentinc/confluent-kafka-python

Docs: http://docs.confluent.io/current/clients/index.html


Kafka Version: 0.8.x, 0.9.x, 0.10.x, 0.11.x

Maintainers: Confluent.io

License: Apache 2.0 


-------------

Pure Python implementation with full protocol support. Consumer and Producer implementations included, GZIP, LZ4, and Snappy compression supported.

http://github.com/dpkp/kafka-python


Kafka Version: 0.8.x

Maintainer: Dana Powers
License: Apache v.2.0

 

, 0.9.x, 0.10.x, 0.11.x, 1.0.x

Maintainer: Dana Powers
License: Apache 2.0


-------------

Python driver with full protocol support, Pythonic API, and managed balanced consumer implementation. GZIP and Snappy compression supported.
https://github.com/Parsely/pykafka
Kafka Version: 0.8.x,0.9.x,0.10.x
Maintainer: Parse.ly
License: Apache v22.0

 


-------------

Protocol support for Kafka 0.7 in Python. GZip and Snappy compression supported

Kafka Version: 0.7.x

Maintainer: David Arthur
License: Apache v.2.0

https://github.com/mumrah/kafka-python/tree/0.7

-------------

Also:
https://github.com/dsully/pykafka

 


Kafka Version: 0.7.x

Maintainer:: Dan Sulley, LinkedIn
License: Apache 2.0

-------------

Python client from Disqus:
https://github.com/getsamsa/samsa 


Kafka Version: 0.7.x

Maintainer:: Keith Bourgoin, Parse.ly
License: Apache 2.0

-------------

Python client from Urban Airship: https://github.com/urbanairship/pykafkap

 


Kafka Version: 0.7.x

-------------

Python client from Datadog: https://github.com/datadog/brod
(Producer, Simple Consumer, ZK-Consumer) 


Kafka Version: 0.7.x

-------------

...

https://github.com/Shopify/sarama 

-------------

confluent-kafka-go: Confluent's Kafka client for Golang wraps the librdkafka C library, providing full Kafka protocol support with great performance and reliability.

The Golang bindings provides a high-level Producer and Consumer with support for the balanced consumer groups of Apache Kafka 0.9 and aboveEnhanced Go Kafka consumer and producer implementations.

Kafka Version: 0.8.x, 0.9.x,  0.10.x, 0.11.x

Maintainer: ConfluentMaintainer: Big Data Open Source Security

License: Apache v2.0

https://github.com/stealthlyconfluentinc/go_kafka_clientconfluent-kafka-go

Docs: http://docs.confluent.io/current/clients/index.html 

-------------

A pure Go implementation of the low level Kafka API.

Pure Go implementation providing full protocol support, great performance, and support for all client facing KIPs.

Kafka Kafka Version: 0.8.x0+

Maintainer: Big Data Open Source SecurityTravis Bischel

License: Apache v2.0 MIT

kafka-go repo

Docs: https://pkg.go.dev/github.com/stealthly/siesta twmb/kafka-go/pkg/kgo

-------------


Erlang

-------------

A fast pure Go Kafka implementation with a clean API.

 

Kafka Version: 0.8.x

Maintainer: OptioPay
License: MIT

Kafka client library in Erlang. Full support for 0.9+ consumer protocol, very efficient producer implementation.

https://github.com/optiopayklarna/kafka

-------------

https://github.com/nuance/kafka

https://github.com/jdamick/kafka.go

brod


Kafka Version: 0.9.x, 0.10.x

Maintainers: Ivan Dyachkov (Klarna AB), Shi Zaiming (Klarna AB)

License: Apache 2.0Kafka Version: 0.7.x


-------------

Erlang

-------------

Kafka client library in Erlang. Full support for 0.9+ consumer protocol, very efficient producer implementation.

...

A minimal, high-performance Kafka client in Erlang

https://github.com/

...

helpshift/

...

ekaf

 


Kafka Version: 0.

...

8.x

...

Maintainer: Helpshift
License: Apache

...

v2

 


-------------

A minimal, high-performance Kafka client in Erlangerlkafka is a kafka client written in erlang

https://github.com/helpshift/ekaf

Kafka Version: 0.8.x

Maintainer: Helpshift
License: Apache v2

 

-------------

erlkafka is a kafka client written in erlang

https://github.com/milindparikh/erlkafka.git


Kafka Version: 0.7.x

Maintainer: Milind Parikh
*License: BSD, LGPL

Also:
https://github.com/wooga/kafka-erlang 

PowerShell

.NET

-------------

Pure C# client with full protocol support.  Includes consumer, producer,

...

PowerShell client supporting Producer, Consumer, Admin; plus from PowerShell it is possible to run all tools available.

Full list of available cmdlets on KNet PowerShell usage

Kafka Version:

...

3.

...

6.

...

0

Maintainer:

...

MASES s.r.l.

License: Apache 2.0

https://github.com/

...

masesgroup/

...

 

...

KNet

Direct link: https://www.powershellgallery.com/packages/MASES.KNetPS


.NET

-------------

...

KNet is a comprehensive .NET

...

suite for

...

Apache Kafka providing all features: Producer, Consumer, Admin, Streams, Connect, backends (ZooKeeper and Kafka).

Kafka Version: 3.6.0

Maintainer: MASES s.r.l.

Kafka Version: 0.8.x

Maintainer: ExactTarget

License: Apache 2.0

https://github.com/

...

masesgroup/

...

KNet


 

-------------

rdkafka-dotnet is a C# client for Apache Kafka based on librdkafkaA fully featured .NET client for Apache Kafka based on librdkafka (a fork of rdkafka-dotnet).

Kafka Version: 0.8.x, 0.9.x, 0.10.x, 0.11.x

Maintainer: Confluent Inc. (original author Andreas Heider)Maintainer: Andreas Heider

License: BSD (2 clause)Apache 2.0

https://github.com/confluentinc/ahconfluent-/rdkafkakafka-dotnet

 


-------------

.Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes. The project also offers balanced consumer implementation. The project is a fork from ExactTarget's Kafka-net Client.

 

Pure C# client with full protocol support.  Includes consumer, producer,

lower level components and gzip support (no snappy)


Kafka Kafka Version: 0.8.x, 0.9.x

Maintainer:  MicrosoftJames Roland 

License: Apache 2.0

https://github.com/MicrosoftJroland/Kafkanetkafka-net

 


-------------

C# client, asynchronous, all 3 compressions supported (read and write), tracks leader partition changes transparently, long time in production.

 

This is a .NET implementation of a client for Kafka using C# for Kafka 0.8. It provides for an implementation that covers most basic functionalities to include a simple Producer and Consumer.

Kafka Version: 0.8.x

Maintainer:  Vadim ChekanExactTarget

License: Apache - 2.0

https://github.com/ExactTargetDev/ntent-ad/kafka4net kafka-net


-------------

kafka-sharp - "High Performance" .NET Kafka Driver

 

.Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes. The project also offers balanced consumer implementation. The project is a fork from ExactTarget's Kafka-net Client.


Kafka Version: 0.8.x, 0.9.x

Maintainer: CriteoMicrosoft

License: Apache 2.0

https://github.com/criteoMicrosoft/kafka-sharpKafkanet

 


-------------

 

Clojure

-------------

Fast kafka api for JVM languages implemented in clojure.

 

C# client, asynchronous, all 3 compressions supported (read and write), tracks leader partition changes transparently, long time in production.


Kafka Version: 0.8.x

Maintainer: https://github.com/gerritjvvVadim Chekan
License: Apache-2.0
https://github.com/gerritjvv/kafka-fastntent-ad/kafka4net


-------------

Wrapper to the Java API for interacting with Kafka

 

kafka-sharp - "High Performance" .NET Kafka Driver


Kafka Version: 0.8.x

Maintainer: https://github.com/pinglesCriteo

License: Apache 2.0

https://github.com/pinglescriteo/clj-kafka/  kafka-sharp


-------------

Kafka clojure client library

 kafunk - Fully-featured, native F# client which works on .NET Core

Kafka Version:  00.8.x, 0.9.x, 0.10.x

Maintainer:  Pierre-Yves Ritschard Jet.com

License: MIT
Code, DocumentationApache 2.0

https://github.com/jet/kafunk


------------- 


Clojure

Ruby

-------------

Pure Ruby, Consumer and Producer implementations included, GZIP and Snappy compression supported. Ruby 1.9.3 and up (CI runs MRI 2.0, JRuby and Rubinius).

Kafka Version: 0.8.x

Fast kafka api for JVM languages implemented in clojure.


 

Kafka Version: 0.8.x

Maintainer: https://github.com/gerritjvv

License: Apache 2.0Maintainer: Bob Potter
License: MIT

https://github.com/bpotgerritjvv/poseidonkafka-fast

 


-------------

Karafka - Microframework used to simplify Apache Kafka-based Ruby applications development.Wrapper to the Java API for interacting with Kafka


 

Kafka Version: 0.8.x

Maintainer: Maciej Mensfeld
Licence: MIThttps://github.com/pingles

License: Apache 2.0

https://github.com/karafka/karafkapingles/clj-kafka/ 


-------------

Kafka clojure client library


 ruby-kaka - A Ruby client library for the Kafka distributed log system

Kafka Version: 0.8.x

Maintainer: Zendesk/ Daniel Schierbeck

Licence: Apache 2.0

Pierre-Yves Ritschard

License: MIT

Code, Documentation

-------------


Ruby

https://github.com/zendesk/ruby-kafka

-------------

JRuby wrapper for producers and consumers of the existing API

 Karafka - Ruby and Rails multi-threaded efficient Kafka processing framework. Based on librdkafka.


Kafka Version: 0.8.x 1.0+

Maintainer: Joseph Lawson Maciej Mensfeld

License: Apache 2LGPL-3.0

https://github.com/joekiller/jruby-kafka karafka/karafka


-------------

https://github.com/acrosa/kafka-rb

 

WaterDrop - Standalone Karafka library for producing Kafka messages . Based on librdkafka.


Kafka Version:

...

1.0+

Maintainer:

...

 Maciej Mensfeld

License:

...

MIT

https://github.com/karafka/waterdrop


-------------

Event machine client:
https://github.com/groupme/em-kafka

 

Kafka Version: 0.7.x

-------------

JRuby Event stream processor

rdkafka - Modern low-level Kafka client library for Ruby based on librdkafka


Kafka Version: 1.0+

Maintainer: Maciej Mensfeld

License: MIT

https://github.com/

...

karafka/rdkafka-ruby


kafkaesque

 

Kafka Version: 0.7.x

-------------

 

 

Node.js

-------------

Kafka-Node is a NodeJS client with Zookeeper integration

Racecar – A simple framework for writing Kafka consumers in Ruby that integrates nicely with Rails. Based on ruby-kafka. 


Kafka Version: 0.8.9.x, 0.10.x, 0.11.x

Maintainer: sohu.com Zendesk / Daniel Schierbeck

License: MITApache 2.0

https://github.com/SOHU-Co/kafka-node/ zendesk/racecar


-------------

DeliveryBoy – A simple way to publish messages to Kafka -node is a pure JavaScript implementation for NodeJS Server with Vagrant and Docker supportfrom Ruby applications. Based on ruby-kafka.

Kafka Version: 0.8.9.x, 0.10.x, 0.11.x

Maintainer: Zendesk / Daniel Schierbeck

License wurstmeister
License: Apache 2.0

https://github.com/wurstmeister/node-kafka-0.8-plus

 

zendesk/delivery_boy

-------------


Node

-kafka is a node

.js

wrapper for the C library librdkafka 

Kafka Version: 0.8.x

Maintainer: Sutoiku

License: MIT

https://github.com/sutoiku/node-kafka

 

-------------

kafka-java-bridge is a Nodejs wrapper for the JAVA high level kafka 0.8. consumer API

The node-rdkafka library is a high-performance NodeJS client for Apache Kafka that wraps the native librdkafka library. All the complexity of balancing writes across partitions and managing (possibly ever-changing) brokers should be encapsulated in the library.

https://github.com/Blizzard/node-rdkafka 

Kafka Version: 0.8.x9, 0.10

Node.js >= 4

Maintainer: LivePersonInc Blizzard.com

License: MIThttps



://www.npmjs.com/package/kafka-java-bridgehttps://github.com/LivePersonInc/kafka-java-bridge

 

-------------

Low-level protocol support in node.js.

https://github.com/cainus/Prozess
https://npmjs.org/package/prozess

 

Kafka Version: 0.7.x

Maintainers: 
Gregg Caines
Eric lee
License: MIT

Kafka-Node is a NodeJS client with Zookeeper integration


Kafka Version: 0.8.x

Maintainer: sohu.com

License: MIT

https://github.com/SOHU-Co/kafka-node/ 


-------------

Alternate node client from Tagged
https://github.com/marcuswestin/node-kafka

Kafka-node is a pure JavaScript implementation for NodeJS Server with Vagrant and Docker support.


Kafka Version: 0.8.x

Maintainer: wurstmeister
License: Apache 2.0

Also:
https://github.com/dannycoateswurstmeister/franznode-kafka

 

Kafka Version: 0.7.x-0.8-plus


-------------

 

Proxy (HTTP REST, etc)

...

kafka-java-bridge is a Nodejs wrapper for the JAVA high level kafka 0.8. consumer API


Kafka Version: 0.8.x

Maintainer: LivePersonInc

License: MIT

https://www.npmjs.com/package/kafka-java-bridge

https://github.com/LivePersonInc/kafka-java-bridge


---------

...

----

Low-level protocol support in node.js.

Dory is a producer daemon that supports clients in various programming languages.  Clients send messages to Dory using local interprocess communication.  Dory then takes full responsibility for reliable message delivery.  Monitoring infrastructure can query Dory's web interface for JSON-based status and data quality reports.

...

https://github.com/

...

cainus/Prozess
https://npmjs.org/package/prozess


Kafka Version: 0.

...

7.x

...

Maintainers: 

...


Gregg Caines
Eric lee
License:

...

 MIT


-------------

Apache Kafka HTTP Endpoint for producing and consuming messages from topics

Kafka Version: 0.8.x

Maintainer: Big Data Open Source Security LLC
License: Apache 2.0

...

Alternate node client from Tagged
https://github.com/marcuswestin/node-kafka

Also:
https://github.com/

...

dannycoates/

...

franz-kafka

...


Kafka Version: 0.7.x

...

-------------


Proxy (

...

HTTP REST, etc)

-------------

Dory is a producer daemon that supports clients in various programming languages.  Clients send messages to Dory using local interprocess communication.  Dory then takes full responsibility for reliable message delivery.  Monitoring infrastructure can query Dory's web interface for JSON-based status and data quality reports.


https://github.com/dspeterson/dory


Kafka Version: 0.8.x, 0.9.x, 0.10.x, 0.11.x

Maintainer: Dave Peterson
License: Apache v.2.0

-------------

Apache Kafka HTTP Endpoint for producing and consuming messages from topics


Kafka Version: 0.8.x

Maintainer: Big Data Open Source Security LLC
License: Apache 2.0

https://github.com/stealthly/dropwizard-kafka-http


-------------

(Deprecated) Kafka high level Producer and Consumer APIs are very hard to implement right. Rest endpoint gives access to native Scala high level consumer and producer APIs.


Maintainer: Sasha Klizhentas
License: Apache 2.0

https://github.com/mailgun/kafka-http


-------------

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster.


Kafka Version: 0.8.x, 0.9.x, 0.10.x

Maintainer: Confluent 
License: Apache 2.0

 

Maintainer: Sasha Klizhentas
License: Apache 2.0

...

...


-------------

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster.

 Kafka-Pixy is a local aggregating HTTP proxy to Kafka messaging cluster.


Kafka Version: 0.8.x, 0.9.x, 0.10.x

Maintainer: Confluent 
License: Apache 2.0

...

 Mailgun
License: Apache 2.0

...

https://github.com/

...

mailgun/kafka-

...

pixy

 


-------------

Efficient Kafka REST Proxy for producers

https://github.com/klarna/kastleKafka-Pixy is a local aggregating HTTP proxy to Kafka messaging cluster.


Kafka Version: 0.8.x

Maintainer: Mailgun
License: Apache 2.0

https://github.com/mailgun/kafka-pixy

 

Maintainers: Ivan Dyachkov (Klarna AB), Shi Zaiming (Klarna AB)
License: Apache 2.0


-------------

Efficient Kafka Full featured HTTP REST Proxy for producersKafka in Elixir with fine grained JWT-based authentication

https://github.com/klarna/kastle kastlex


Kafka Version: 0.8.x

Maintainers: Ivan Dyachkov (Klarna AB), Shi Zaiming (Klarna AB)
License: Apache 2.0

 

 


Perl

-------------

 

Pure Perl, Consumer and Producer implementations included. Zookeeper
integration. GZIP and Snappy compression supported.

 


Kafka Version: 0.8.x
Maintainer: Sergey Gladkov
License: Artistic License

https://github.com/TrackingSoft/Kafka
http://search.cpan.org/~sgladkov/Kafka/lib/Kafka.pm 

 


 

stdin/stdout

-------------

...

Maintainer: Magnus Edenhill
License: 2-clause BSD

 


 

 

PHP

-------------

PHP extension built on librdkafka 


Kafka Version: 0.8.x

Maintainer:  Elias Van Ootegem
License: MIT 


https://github.com/EVODelavega/phpkafka 


-------------

PHP client based on librdkafka

...

Maintainer: Arnaud Le Blanc, for Mention.com
License: MIT 


https://github.com/arnaud-lb/php-rdkafka

 


-------------

PHP library with Consumer (simple and Zookeeper-based), Producer and compression support (release notes).

https://github.com/quipo/kafka-php

 


Kafka Version: 0.7.x

Maintainer: Lorenzo Alberton
License: Apache v.2.0

...

https://github.com/dastra/log4php-kafka

 


------------- 


Rust

-------------

Pure Rust implementation with support for Metadata, Produce, Fetch, and Offset requests. Supports Gzip and Snappy compression

 


Kafka Version: 0.8.x

Maintainer:  Yousuf Fauzan
License: MIT 

Code
Documentation 


Alternative Java

-------------

...

Krackle is an optimized Kafka client built by Blackberry. 


Kafka Version: 0.8.x

Maintainer: Blackberry

...

https://github.com/blackberry/Krackle/blob/dev/LICENSE 


Storm

-------------

Port of Apache storm-kafka to >= 0.8.0

...

https://github.com/wurstmeister/storm-kafka-0.8-plus 


Scala DSL 

-------------

A DSL for Scala developers to produce and consume messages with Kafka.

 


Kafka Version: 0.8.x

Maintainer: Big Data Open Source Security LLC
License: Apache 2.0

https://github.com/stealthly/scala-kafka

 


Clojure

-------------

https://github.com/pingles/clj-kafka

https://github.com/miniway/kafka-clj


Kafka Version: 0.7.x


Swift

-------------

Pure Swift implementation of Producer + Simple and High-Level Consumer 


Kafka Version: 0.79.x

Maintainer: Kellan Cummings, Luke Lau
License: MIT


https://github.com/kellanburket/franz


Client Libraries Previously Supported

...