Versions Compared

Old Version 2

changes.mady.by.user Greg Harris

Saved on

compared with

New Version 3

changes.mady.by.user Viktor Somogyi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Consumer offsets should be replicated to the target cluster synchronously, such that the offset arrives before the commit record (does this require 2PC? can this be configured in the replication link?)
  • An exactly-once application which fails-over from source to target clusters should maintain it's exactly-once semantics.

Remote and Local Logs

  • Remote log replication should be controlled by the second tier's provider.
  • Remote logs can be referenced if not replicated by the second tier's provider so that replicated Kafka topics reference the source remote log storage.

Networking

  • The network path between Kafka clusters is assumed to be less reliable than the intra-cluster network, and have more strict constraints
  • Network connections for replication links can proceed either source→target or target→source to allow one cluster to be behind a NAT (can we support NAT punching as a first-class feature, to allow both clusters to be behind NAT?)
  • Allow for simple traffic separation of cross-cluster connections and client/intra-cluster connections (do we need to connect to something other than the other cluster's advertised listeners?)

...