Avoid the queuing of dropped events by the primary gateway sender when the gateway sender is stopped
To be Reviewed By: July 9th16th, 2020
Authors: Alberto Gomez (alberto.gomez@est.tech)
Status: Draft Draft | Discussion | Active | Dropped | Superseded
...
As described above, dropped events in the primary gateway sender are stored in a member variable. It is out of the scope of this RFC to change how those events are stored.
Solution 1 (original proposal, deprecated)
The solution proposes to change the primary gateway sender so that it does not store dropped events when it is stopped explicitly (not while starting). The reason is that these events could never end in the queue of any secondary gateway sender and will use memory unnecessarily.
In order to do so, it is proposed to add a new boolean member variable (mustQueueDroppedEvents)
to the AbstractGatewaySender
that will tell if the primary gateway sender must store dropped events or not.
...
- Code will be added at the end of the current
stop gateway sender
gfsh command which will setmustQueueDroppedEvents
to false (after the gateway senders have been stopped). - Code will be added at the beginning of the current
start gateway sender
gfsh command which will setmustQueueDroppedEvents
to true (before the gateway senders are started).
In case the GatewaySender
Java API is used to start/stop gateway senders, in order to get the new behavior, given that the scope of the start/stop methods is the VM on which it is invoked, it will be necessary to set the mustQueueDroppedEvents
accordingly (to true before starting all sender instances and to false after stopping all sender instances) on every VM. To set the value of the variable, the GatewaySender interface will offer the following new method: setMustQueueDroppedEvents(boolean mustQueue)
. If the new method is not used, the legacy behavior will prevail except if the gateway sender is started manually in which case dropped events will not be queued.
A draft PR of the solution can be found here: https://github.com/apache/geode/pull/5348
Changes and Additions to Public Interfaces
No changes to public interfaces are proposedTwo new methods: setMustQueueDroppedEvents(boolean)
and mustQueueDroppedEvents()
will be added to the GatewaySender
public interface.
Performance Impact
As the proposal implies changing the implementation of the gfsh start gateway sender
and the stop gateway sender
gfsh commands to be done in two steps, these commands may be slightly slower but not significantlyalthough not significantly.
Backwards Compatibility and Upgrade Path
The proposal does not affect the rolling upgrade and has not impacts in the regular rolling upgrade process.
Solution 2 (new proposal, agreed after discussions on this RFC )
The solution consists of, instead of storing dropped events in `tmpDroppedEvents` to later send batch removal messages when the primary gateway sender is not started, try to send the batch removal message when the event to be dropped is received. That way, when the sender is stopped for a long time and there are events coming, the memory of the `AbstractGatewaySender` will not grow with entries in the `tmpDroppedEvents` member.
In order to send the batch removal message directly, the `eventProcessor` for the `AbstractGatewaySender` must have been created. If it is not yet created because the sender was created with manual start set to true, while receiving events to be dropped, they will be stored in `tmpDroppedEvents` as there is no other choice. Nevertheless, in order to consume less memory, the event stored could be a simplified event containing only the necessary information to handle it.
A draft PR of the solution can be found here: https://github.com/apache/geode/pull/5486
Changes and Additions to Public Interfaces
No changes.
Performance Impact
No impacts foreseen.
Backwards Compatibility and Upgrade Path
The proposal does not affect the rolling upgrade and has not impacts in the regular rolling upgrade process.
...