Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

REST server accepts produce requests for specific topics or partitions. It internally uses java producer instance to write messages into Kafka. 

Consumer API

REST server Proxy uses the new consumer API to consume the messages from the subscribed topic on behalf of the
specified consumer group. Consumer instances are stateful and tied to a particular Rest server instance.
A full URL is provided when the instance is created and it should be used to construct any subsequent requests.
When a message is consumed on behalf of a consume group for the first time, then Kafka Consumer instance joins
the consumer group
and subscribes to the topic. All Consumer instances that are currently members of that group
and subscribed to that topic divide
topic partitions among themselves. If a Consumer instance has not consumed
from a particular topic on behalf of a particular
consumer group for configured interval (normally large interval),
then it unsubscribes from the topic on behalf of that group. This is for cleaning unused consumer instances due to dead clients.
Offset commit can be either
automatic or manual as requested by the user.

We can also retrieve the messages for a consumer, from a specific partition,
starting with an offset.

We also want to take community opinion on other ways of implementing consumer group

...

functionality.
Admin API and Security Integration:

...