Page History

The Java broker currently has a mixed model for storage of configuration, where some things are held in (multiple) XML files, others are held in a VirtualHosts VirtualHost's durable (message) storage, and some can even be held in both. This can create challenges in determining which configuration source should be use used to retrieve particular information, and reconciling which configuration source should dominate in the event that overlaps occur.

The following page is intended to document analysis and design aspects of efforts to unify the configuration model of the Java broker and provide it a single consistent storage model for configuration, held in a (durable) configuration store. The design for the new configuration store is in New design for the Java Broker configuration.

This is core to an overall effort to improve usability of the product and increase its suitability for use in Cloud environments. Components of this work include removal of the XML based configuration, addition of a new configuration model for broker entities, consistent storage of that model information, and new management functionality enabling a self-service style operation of remote broker instances via means such as browser based management and a REST API in addition to improved JMX support.

Table of contents

Configuration Entities

Relationships

...

Time To Live for configured objects represents how long an object will live once the conditions of its Lifetime Policy are no longer satisfied (e.g. a queue with an auto-delete policy could be deleted a certain amount of time after consumer/session closes rather than immediately, or an otherwise permanent queue could be deleted after being idle for a configured period of time).

Configuration Attributes and Statistics

Each of the ConfiguredObjects will have a number of attributes associated with them, including the several noted above that apply to all objects, and a group of other type specific (queue, exchange etc) and possibly instance specific (priority queue, LVQ, sorted queue etc) attributes. Additionally each ConfiguredObject will separately maintain a number of statistics (which are not persisted) which track elements of its ongoing usage since broker startup.

The below tables (generated from the attached spreadsheet) map out the base attributes and statistics each object type will have:

Overall work plan

1. Update BDB+Derby message stores to use new configuration storage model, representing ConfiguredObjects in a more generic fashion through storage attributes as JSON rather than specific table columns as at present.
   • Maintain existing store interface for now, use a new layer shared between BDB+Derby to convert existing objects into the JSON format stored by the new store schema, allowing later change of the store interface using the same on-disk format.
   • Update message content and metadata references to queues to use ids rather than names (allowing queues to be renamed).
   • TODO: what about messages for queues that are definied in XML? (Possibly generate UUID based from name (nameUUIDFromBytes(byte[] )
2. Refine store upgrade process using a model whereby the broker automatically performs the upgrades between each store version upon startup (inc multiple upgrades spanning version gaps).
   • Isolate individual upgrade steps, remove version numbers form the tables and have a version table etc (See Robs patch for example)
   • Convert existing messages to use new id-reference to their associated queue.
3. Complete implementation of new plugin based JMX MBean adaption layer.
   • VirtualHost, Exchange, Queue etc
   • Updating of tests as necesssary and/or creation of new unit tests to replace old system tests where possible.
   • UserManagement, LoggingManagement MBeans still TBC
4. Extract the Operational Logging from the core broker and define as a plugin listening for configuration events.
5. (Complete) Definition of the REST management interface
6. (Complete) Implementation of the REST management interface
7. Design the desired layout for the new web management UI
8. Get the basic web management ui running
9. Make the ui 'pretty'

This is also covered diagramatically in the attached PDF

Notes

• Alternate Exchanges: Although the alternate-exchange is listed as an attribute of the Queue and Exchange above, it probably shouldnt be: the 'attribute' for any given Queue or Exchange will simply be the name of its alternate exchange, but this is not somethign we would want to store on the Queue or Exchange in question since we want to be able to rename Exchanges. Additionally, storing the alternate exchange on an object could make recovery difficult as a specific recovery order would be required which may ultimately be impossible to achieve (e.g. there may be exchange-exchange alternate loops). Instead of storing the alternate exchange as an attribute directly, what we really want to do is be able to define a relationship between a given Queue/Exchange and its alternate (not unlike the parent-child relationships ConfiguredObjects will already have). For this we probably want to seperately store an 'alternate' relationship in the store, relating a given Quue/Exchange with its alternate Exchange and which can be applied after creating the ConfiguredObjects during recovery.

• Queues in XML configuration: When doing upgrades (/before we get rid of the configuration file), it is possible there will be queues definied in the configuration file (rather than the store) which have messages. IDs will need made for these queues to allow matching id-referenced message data in the store. We will probably need to use something like a UUID generated from the queue name as the ID, eg using nameUUIDFromBytes(byte[] name).

• User vs Store configuration: The current intension is for the storage of configuration to be through JSON, however this is also the likely avenue to be taken for users being able to extract and replay their configuration in future through the management interfaces. There is a key difference in that user-facing configuration is still likely to use names for Queue/Exchange/VirtualHost etc configuration, whereas the store is going to mainly refer to objects by their ID to allow renaming etc.