Versions Compared

Old Version 1

changes.mady.by.user Joel Koshy

Saved on

compared with

New Version Current

changes.mady.by.user Joel Koshy

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this approach, producers send metadata about each transaction to a special (replicated) journal log. Each partition of the actual data log also has a corresponding local data journal log. After the producer commits the transaction the brokers copy the data from the local data journal log (locally) into the actual data log.

Image Added

Phase 1 (journaling)

  • If the producer needs strict ordering then it should form a transactional group (G) with the other producers that may participate in the transaction. This could be a configured property. Otherwise it can use a random group G.Obtain a transaction ID (from any broker). These transaction IDs may be generated off a ZooKeeper queue. This is a blocking call and we could return more than one transaction ID that the producer can use in subsequent transactions, but we will anyway need to block in the commit (further down).

  • Send BEGIN-TxId to the journal partition for its group based on some hash fn - i.e., J-fn(G).

  • Send (pipelined) messages for the transaction to the respective leaders of the journal partitions that correspond to the data partitions that comprise the transaction.

  • Send COMMIT-TxId to the J-fn(G). A successful (non-error) response to this COMMIT message indicates to the producer that the transaction will be committed.

...