THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Some of the good practices and ideas will be borrowed from existing tools.
Proposed Changes
Rest Server/Proxy:
The REST Proxy is a separate server, sitting between Kafka cluster and client applications. It is wrapper around existing client libraries.
Request/Responses supports JSON format with embedded data format (JSON and Base64 encoded strings). Proxy uses vendor specific
content types in Content-Type and Accept headers to make the format of the data explicit. This approach is borrowed from Confluent Rest Proxy.
Supported Content-Types are: application/vnd.kafka.binary.v1+json, application/vnd.kafka.json.v1+json
Rest Proxy can be easily scaled by deploying multiple proxy instances. This way we can spread the load across multiple proxy instances.
Producer API:
REST Proxy accepts produce requests for specific topics or partitions. It internally uses java producer instance to write messages into Kafka.
Consumer API:
Admin API and Security Integration:
...