Distributed Entity Cache Clearing Mechanism

Why and when use it?

The distributed cache clearing mechanism is needed when you have multiple servers in a cluster that are sharing a single database. When one entity engine has either a create, update or delete operation that goes through it, it will clear its own caches. But it can also sent out a message to the other servers in the pool to clear their caches.

This is a feature that runs through the service engine which operates on top of the entity engine. When you're doing a distributed cache clear it will result in service calls. In most cases you will use Java Messaging Service, to send a message to a JMS server that can then be distributed to other servers in the cluster.

How to set it?

To keep it simple we will only set the mandatory values. There are other options which are covered by defaults.

The Entity Engine

This is the easiest part, for a given delegator you only have to set its distributed-cache-clear-enable attribute to "true" (false by default)

The Service Engine

The location of the JMS definition is in the service-engine.xml file. By default you set a jms-service of name "serviceMessenger". You define there a JMS server with its name, a jndi name and a topic name. To make as less changes as possible we use "default" for the server name and set the values in the jndi.properties file. I could have also set a server name in jndiservers.xml but my catch phrase is "the less changes the better". This is the service-engine.xml setting, same on each servers

<!-- JMS Service Active MQ Topic Configuration (set as default in jndi.properties, the less changes the better) -->
<jms-service name="serviceMessenger" send-mode="all">
    <server jndi-server-name="default"
        jndi-name="topicConnectionFactory"
        topic-queue="OFBTopic"
        type="topic"
        listen="true"/>
</jms-service>

I decided to use Apache ActiveMQ as JMS server and to simply set these properties in the jndi.properties files (commenting out the OOTB default):

java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory
java.naming.provider.url=tcp://172.18.7.4:61616
topic.OFBTopic=OFBTopic
connectionFactoryNames=connectionFactory, queueConnectionFactory, topicConnectionFactory

...

You can then monitor ActiveMQ using the Web Console by pointing your browser at http://localhost:8161/admin/ and then topics page

Single point of failure

The setting above is sufficient in a staging environment but is a single point of failure in a production environment. So we need to create a cluster of ActiveMQ brokers. Since they should not consume much resources (only 256MB of memory at max and not much CPU cycles), we can put each instance on the same machines than the OFBiz instances.

...

It's fairly simple to set through JNDI. For this we only need to replace in jndi.properties files

java.naming.provider.url=tcp://172.18.7.4:61616

by

java.naming.provider.url=failover:(tcp://172.18.7.4:61616?soTimeout=60000,tcp://172.18.7.5:61616?soTimeout=60000)?randomize=false

...

1. set advisorySupport="false" for the broker (exceptif you want to use advisroy messages)

2. use transport.soTimeout=60000 and set enableStatusMonitor="true" for Openwire connector

   Code Block
   <broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.base}/data" destroyApplicationContextOnStop="true" advisorySupport="false"> .... <transportConnector name="openwire" uri="tcp://0.0.0.0:61616?transport.soTimeout=60000" enableStatusMonitor="true"/>

A possible pitfall

This issue should be resolved if you use a recent version (say after mid 2011): we have now the proper slf4j jar files in the classpath to redirect logging to log4j.

This section can be bypassed but might help in case of troubles
I first installed ActiveMQ 5.5.0 on my developement machine on XP. When I ran OFBiz I got this non blocking error

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

...