Page History

This is an un-ordered compendium of projects and project ideas.

Java Client Re-write

There are a number of limitations in the current JVM clients outlined in the above link.

Security

Authentication (users) and authentication (permissions) at the topic level.

Quotas

Provide a quota mechanism to limit throughput and # of requests on a per-topic and per-user basis.

Non-java Clients

We want to improve the client libraries for the major languages (ruby, python, c++, etc). Some of these don't yet have a 0.8 compatible library available and for others the client is somewhat limited and could be improved.

...

Now that we have replication it would be possible to implement exactly-once producer semantics.

Offset Storage

We have added APIs for clients to manage offsets but we still are storing them in Zookeeper under the covers. This is unfortunately not at all scalable for high commit rates for larger numbers of consumers. There is a proposal for implementing a more scalable store for these.

Log Slurper

For people who want to publish Kafka feeds for existing applications that produce log files it might be nice to have something more sophisticated than the console-producer. This would be a process that ran in the background and tailed log directories and read and published formatted messages.

...