You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Current »

I am working on a proposal to satisfy the Restful Proxy project listed on the Projects page. (http://incubator.apache.org/kafka/projects.html)

My working knowledge of Kafka is about 3 hours old and I have actually written zero code yet, it is my job to do so for Box so I will update this as my knowledge increases. There may be some fundamental issues with this proposal at this time.

General Idea

Create a Restful API which will allow producers and consumers to communicate with a Kafka server via HTTP calls

Consumers

URL Format:

GET /messages/<topic>/<consumer>/<offset> - Retrieve the messages for a consumer, from the topic, starting with an offset

DELETE /messages/<topic>/<consumer>/<offset> - ACK consumption of message for a consumer, topic, offset

Producers

PUT /messages/<topic>

Request Body will become the message

Known Missing Items:

  • I know that this does not take into account partitions.
  • It would be nice to be able to implement the async producer and have an HTTP callback happen from within the RESTServer (producer specifies a URL to be requested as callback)
  • If an HTTP call back wasn't available, have the service return a task ID (MD5 of the message or something), and then allow the client to query the server to see if the callback from Kafka has happened yet.
  • No labels