Gw sender to deliver transaction events atomically to receivers
To be Reviewed By: April 1st, 2020
Authors: Alberto Gomez (alberto.gomez.reyes@estgomez@est.tech)
Status: Draft Draft | Discussion | Active | Dropped | Superseded
Superseded by: N/A
...
If there is a network split in a WAN deployment when the last batch received from a sender does not contain all the events for one or more transactions, the data in the receiving side will be inconsistent while the network split remains. Inconsistent data due to transactions not applied atomically applying an incomplete number of the operations for a transaction could be very problematic for clients.
...
It is also assumed that the events to be delivered atomically must be so by means of the same gateway sender. Therefore, the deployments that require this feature should configure replication such that the regions to which the data in the transactions belong must be collocated and must have the same gateway sendergateway senders. Additionally, if serial gateway senders are used, they must be configured with just one dispatcher thread given that it would be impossible to fulfill the requirement if each event of a transaction is handled by a different dispatcher thread.
Solution
In order for the gateway senders to deliver batches that contain all the events for the transactions the following is proposed:
- After transaction commit and before events are sent to the gateway sender, mark the last event of the transaction (a new boolean attribute “lastEventInTransaction” may be added to EntryEventImpl and GatewaySenderEventImpl)
- In the process of creating a batch, the gateway sender must keep track of all the transactions involved in the events it is putting in the batch.
- Once a batch is due to be sent (due to maximum configured size is reached or time), the gateway sender must check:
- If for every transaction to which events on the batch belong, the last event has been received, then the batch may be sent.
- Otherwise, two options are proposed:
- a) Keep reading events from the queue and put them in the batch until the above condition is fulfilled. Then send the batch. This option has the problem that the size of batches could grow a lot beyond the maximum size in situations of big transactions and high number of them.
- b) Go to the queue and get selectively the events for the transactions in the batch that do not have all the events. Once these events are added to the batch, send it. This option is preferred over option a) because it should avoid the problem of batches much bigger than the maximum size configured.
Given that this behavior requires a given configuration of replication as mentioned above, it is proposed to make it configurable by means of a new parameter to the "create gateway sender" command (--group-transaction-events) and that is not activated by default.
Other solutions explored to this problem that have been discarded are:
- Send a new type of event inside batches , -event transactions events, that will contain all the events of a given transaction). This solution, did does not seem to fit well with the current code base as events are triggered on a region basis while transactions could contain events from several regions. The changes in the code required seemed to be much greater than the ones with the preferred solution.
- Make sure in the gateway receiver that events for a transaction are only applied if all the batches where they have been delivered, have been received. This solution seemed to be more complex than the one preferred and did not provide extra benefits.
Changes and Additions to Public Interfaces
No changes to public interfaces are proposedA new method to GatewaySenderFactory, setGroupTransactionEvents() to create gateway senders with support for this new feature must be added.
Performance Impact
The performance of gateway senders may be impacted as there will be an extra cost of keeping track of transactions in batches and also of retrieving selectively events from the queue (if this option is selected).
...
The gateway senders will behave as before if the new parameter --group-transaction-events is not used.
Versioned support for GatewaySenderEventImpl must be added due to the new field added.
Prior Art
As said above, with the current behavior of gateway senders, on the event of network splits, clients will have to deal with data inconsistencies provoked by events not delivered atomically for the time the network split is in place.
...