Highly Available Client Event Mechanism

...

One of the key feature of Geode is reliable asynchronous event notificationsnotification. The publish/subscribe systems offer offers a data distribution service where new events are published into the system and routed to all interested subscribers in a reliable manner.

...

When an operation is performed on the cache, the “LocalRegion.notifyBridgeClients()” is called to see if there are interested clients (“generateLocalFilterRoutingdeliver events to the interested clients. The list of interested clients are obtained by calling  “generateLocalFilterRouting()” ) and to deliver the event (and events are queued through “CacheClientNotifier.notifyClients()”).

Based on the interest satisfied by the client, the ClientMessage is created and added to the HARegionQueue of the interested client (“CacheClientProxy.enqueueMessage()”).Once  Once the event is added to the HARegionQueue the cache operation thread returns to the caller.

...

The client events are delivered asynchronously to the clients using MessageDispatcher thread ("CacheClientProxy.MessageDispatcher"). The dispatcher thread peeks and event from the HARegionQueue and delivers it to the client.

...

Client:
Upon receiving the event, the client updates its local cache or invokes the CqListeners (“CacheClientUpdater.processMessages()”).

Also the The client also acknowledges the server periodically with the received event Id (ThreadId, sequence Id); for removal of events from HARegionQueue.

Server:
The server process processes the ack command from client and maintains it in HARegionQueue.ackedEvents map, uses this for message removal. 

Removing Events:

The event from HARegionQueue is removed based on the client acknowledgement (“HAregionQueue.remove()”).

...

The reliable event delivery (in case of node failure) is achieved by subscription redundancy level. Client can configure to have redundant event queues created on multiple server nodes by setting the redundancy level.

When With redundancy is set; the HARegionQueues are created on multiple servers; one of the queue is treated as primary queue and others as secondary queues.

When the cache operation is performed, it is distributed to other nodes; based on the interest subscription the events are added to primary and secondary HARegionQueues (as explained above). The HARegionQueue’s by themselves don’t replicate the events between them.

...

When the message is delivered and successfully removed from primary HARegionQueue, the acknowledgement is sent to other server servers hosting the queues (“QueueRemovalMessage”).

...

During queue initialization, If there are any events satisfying the interest criteria; those events are queued temporarily in “CacheClientProxy.queuedEvents”, once the initialization is over, the events from the temporary queue is added to the HARegionQueue for delivery.

...

The reason for doing this is, as we have seen, in most of the cases clients will have common interests, having a single copy of the client message will reduce the memory significantly.

...