...

• We discussed about the possibility of immediately building a metadata topic to propagate the changes. This seems aligned with the eventual metadata quorum path, but at a cost of blocking the current API migration towards the bridge release, since the metadata quorum design is much more complicated and requires more iterations. To avoid this extra dependency on other tracks, we should go ahead and migrate existing protocols to meet the bridge release goal sooner.
• We also had an almost complete proposal around forwarding request. Just keep it here for future reference. Although the Envelope API provides certain privileges like data embedding and principal embedding, it creates a security hole by letting a malicious user impersonate any forwarding broker. Passing the principal around also increases the vulnerability, compared with other standard ways such as passing a verified token, but it is unfortunately not fully supported with Kafka security. So for the security concerns, we are abandoning the Envelope approach and fallback to just forward the raw admin requests.

==================================================== ============== Attached Old Proposal  ==========================Start Old Proposal  ======================================================== 

New Envelope RPC

We are also going to add a new RPC type to wrap the original request during the forwarding. We will make corresponding changes to `ApiMessageTypeGenerator` class to recognize the new field `Header` and `ApiMessage` during the auto generation. This request will be fully wrapping an older version client request, including its header, security information, actual data fields, etc. The request requires ClusterAction on CLUSTER.

Routing Request Security

 and audit logging purpose, we proposed to add the following fields:

...

1. For next 2.x release:
   1. Get new admin client forwarding changes
   2. Get the Envelope RPC implementation
   3. Get the forwarding path working and validate the function with fake principals in testing environment, without actual triggering in the production system
2. For next 3.0 release:
   1. Introduce serializability to PrincipalBuilder
   2. Turn on forwarding path in production and perform end-to-end testing

======================================================= =========== End Old Proposal  ========================== ======================================================== 

Future Works

We have also discussed about migrating the metadata read path to controller-only for read-after-write consistency. This sounds like a nice improvement but needs more discussions on trade-offs between overloading controller and the metadata consistency, also the progress of Raft quorum design as well.

...