Table of Contents |
---|
StatusStatus
Current state: Under Discussion DISCARDED
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: KAFKA-3294
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
There are already some open-source REST proxies are available. Confluent Confluent REST Proxy got comprehensive interface.
But we would like to add REST server that many users ask for under Apache Kafka repo.
We want to add Kafka Rest Server to Kafka for the following reasons.
...
Supported Content Content-Types are: application/vnd.kafka.binary.v1+json, application/vnd.kafka.json.v1+json
Producer API
REST server accepts produce requests for specific topics or partitions. It internally uses java producer instance to write messages into Kafka.
...
GET /topics/:topic_name/partitions/:partition?offset=(int)
Description : Consume messages from one partition of the topic.
Parameters:
topic_name (String) - topic name
partition (int) - partition number
offset (int) - offset to fetch
Response:
JSON Object contains response objects
Status Codes:
404 Not Found
Error Code 40401 - topic Not Found
Error Code 40402 - partition Not Found
500 Internal Server Error
Error Code 50001 - Kafka Error
Example request:
GET /consume/test/partitions/1?offset=1 HTTP/1.1 Accept: application/vnd.kafka.binary.v1+json |
Example response:
HTTP/1.1 200 OK Content-Type: application/vnd.kafka.binary.v1+json [ { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 1, }, { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 2, } ] |
TODO
Compatibility, Deprecation, and Migration Plan
This KIP only proposes additions. There should be no compatibility issues.
Rejected Alternatives
Make Kafka Rest Server an external third-party tool
Main Reason for this KIP is to add Rest Server to Kafka. Shipping Kafka Rest Server as part of a normal Kafka release
makes it immediately available to every user that downloads Kafka. Also helps to maintain the version compatibility
between Kafka clients and Rest Server.
Push/Stream messages to end clients:
End clients can register for the listening of events. This can be implemented used Web socket push notifications.
But This is against kafka consumer's pull model. It is good maintain Kafka consumer's semantics.