Page History

...

Authors: Alberto Gomez (alberto.gomez.reyes@estgomez@est.tech)

Status: Draft | Discussion | Active | Dropped | Superseded

Superseded by: N/A

...

If there is a network split in a WAN deployment when the last batch received from a sender does not contain all the events for one or more transactions, the data in the receiving side will be inconsistent while the network split remains. Inconsistent data due to transactions not applied atomically applying an incomplete number of the operations for a transaction could be very problematic for clients.

...

It is also assumed that the events to be delivered atomically must be so by means of the same gateway sender. Therefore, the deployments that require this feature should configure replication such that the regions to which the data in the transactions belong must be collocated and must have the same gateway sendersenders. Additionally, if serial gateway senders are used, they must be configured with just one dispatcher thread given that it would be impossible to fulfill the requirement if each event of a transaction is handled by a different dispatcher thread.

Solution

In order for the gateway senders to deliver batches that contain all the events for the transactions the following is proposed:

• After transaction commit and before events are sent to the gateway sender, mark the last event of the transaction (a new boolean attribute “lastEventInTransaction” may be added to EntryEventImpl and GatewaySenderEventImpl)
  
• In the process of creating a batch, the gateway sender must keep track of all the transactions involved in the events it is putting in the batch.
• Once a batch is due to be sent (due to maximum configured size is reached or time), the gateway sender must check:
  • If  for every transaction to which events on the batch belong, the last event has been received, then the batch may be sent.
  • Otherwise, two options are proposed:
    • a) Keep reading events from the queue and put them in the batch until the above condition is fulfilled. Then send the batch. This option has the problem that the size of batches could grow a lot beyond the maximum size in situations of big transactions and high number of them.
    • b) Go to the queue and get selectively the events for the transactions in the batch that do not have all the events. Once these events are added to the batch, send it. This option is preferred over option a) because it should avoid the problem of batches much bigger than the maximum size configured.

...

Changes and Additions to Public Interfaces

No changes to public interfaces are proposedA new method to GatewaySenderFactory, setGroupTransactionEvents() to create gateway senders with support for this new feature must be added.

Performance Impact

The performance of gateway senders may be impacted as there will be an extra cost of keeping track of transactions in batches and also of retrieving selectively events from the queue (if this option is selected).

...

The gateway senders will behave as before if the new parameter --group-transaction-events is not used.

Versioned support for GatewaySenderEventImpl must be added due to the new field added.

Prior Art

As said above, with the current behavior of gateway senders, on the event of network splits, clients will have to deal with data inconsistencies provoked by events not delivered atomically for the time the network split is in place.

...