...
REST Proxy accepts produce requests for specific topics or partitions. It internally uses java producer instance to write messages into Kafka.
Consumer API:
REST Proxy uses the consumer API to consume the messages from the subscribed topic on behalf of the specified consumer group.
When a message is consumed on behalf of a consume group for the first time, then Kafka Consumer instance joins the consumer group
and subscribes to the topic. All Consumer instances that are currently members of that group and subscribed to that topic divide
topic partitions among themselves. If a Consumer instance has not consumed from a particular topic on behalf of a particular
consumer group for configured interval, then it unsubscribes from the topic on behalf of that group. Offset commit can be either
automatic or manual as requested by the user. We can also retrieve the messages for a consumer, from a specific partition,
starting with an offset.
We also want to take community opinion on other ways of implementing consumer group
functionality.
Admin API and Security Integration:
...
GET /consume/test/partitions/1?offset=1 HTTP/1.1 Accept: application/vnd.kafka.binary.v1+json |
Example response:
HTTP/1.1 200 OK Content-Type: application/vnd.kafka.binary.v1+json [ { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 1, }, { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 2, } ] |
GET /topics/:topic/messages?group=<group>
Description : consumes a message from the specified topic on behalf of the specified consumer group
Parameters:
topic (String) - topic name
group (String) - consumer group id
Response:
JSON Object contains response objects
Status Codes:
404 Not Found
Error Code 40401 - topic Not Found
500 Internal Server Error
Error Code 50001 - Kafka Error
Example binary request:
GET /topics/test/messages?group=testgroup HTTP/1.1 Accept: application/vnd.kafka.binary.v1+json |
Example response:
HTTP/1.1 200 OK Content-Type: application/vnd.kafka.binary.v1+json [ { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 1, }, { "key": "a2V5", "value": "dmFsdWU=", "partition": 1, "offset": 2, } ] |
...