Introduction

The Java Broker requires a mechanism that allows a preference, which is owned by a Principal, to be associated with an arbitrary object with the object hierarchy. An example of a preference could be:

a query
a chart
a preferred timezone
a dashboard layout

A preferences will

be owned by a User Principal.
associated with exactly one configured object.
have a unique ID (UUID)
have a visibility which will allow the preference to be private, or shared amongst a group or groups.
normally preferences will be updatable/deletable by the Principal that created it but:
- Special users will be able view/update/delete the preferences of another user. Users with this permission will normally belong to an identity maintainer role.
- Deleting a configured object will cause the associated preferences to be deleted automatically.
Preferences will be durable (survive broker restart)

Preferences associated with a VirtualHost or below need to automatically propagated automatically amongst the nodes within a HA group. As an operator, this means I will be able to create, say, a query against my HA virtualhost and the same query will be available to me no matter where the mastership resides.

Unlike relationships between CO, preferences are loosely associated with model objects. This means that an operator will be able to remove, say a queue, even if preferences are still associated with it, be they belong to the operator himself or another Principal. In this case, the preference becomes orphaned. Orphaned preferences will be automatically purged by the system

The UI will allow an operator to clone a copy of a preference of another which is visible to him, in order that he may change his own copy.

Some preferences may refer to other preferences, for example, a dashboard preference may refer to one or more query or chart preferences.

For some preference types, a unlimited number of instances may be associated to a single configured object, for example, QUERY. For other preference types, the cardinality will be zero or one. An example will be Timezone.

Preferences need to be extensible and allow for the introduction of further preference types The storage mechanism needs to be versioned so that an automated upgrader can translate an older to a newer one.

By convention, we will associate preferences with either the Broker or Virtualhost. The model will permit, preferences to be associated anywhere, but we don't anticipate exploiting this at the moment.

Some other features may 'understand' preferences natively. An example would be the query API, it may be adapted to allow the caller to invoke a query described by a preference directly, without the need for the client side to retrieve the query and then submit it back to the server for execution.

High level design

REST API

The Preference API will be exposed over REST and must be part of the versioned API. To address the preferences the caller will simply append /preferences as the trailing part of the request. As a consequence, no configured object may have an operation called preferences.

The basic form of the URL to operate on the preferences of the current user only will be:

/api/version/<configured object category>/path/to/configured/object/preferences

Users may make preferences visible to other users. The basic form of the URL to access all the preferences visible to the current user only will be as follows.

/api/version/<configured object category>/path/to/configured/object/visiblepreferences

REST API /preferences - Current user preferences

Retrieve

GET will retrieve the preference(s) associated with the identified object that belong to the authenticated user only.

The response will be 200 unless:

The configured object does not exist (response 404)
The preference (by type/name or id) does not exist (response 404)

If no preferences are associated with the configured object the response will be an empty JSON list.

Retrieving one preference by id

The result will be exactly one object.

GET /api/latest/broker/preferences?id=75d5d2a4-0573-11e6-b512-3e1d05defe78
HTTP/1.1 200 Ok
 
{
 id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
 type: "QUERY",
 name: "mypref",
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 },
 owner: "ldap.mycompany.org:kwall"
 createdDate: 123456789,
 updatedDate: 123456789
}

Retrieving one preference by type/name

The result will be exactly one object

GET /api/latest/broker/preferences/query/mypref
HTTP/1.1 200 Ok
 
{
 id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
 type: "QUERY",
 ....
}

Retrieving all preferences by type

The result will be a list of objects.

GET /api/latest/broker/preferences/query
HTTP/1.1 200 Ok
 
[{
  id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
  type: "QUERY",
  ....
 },
 {
  id: "0e046c3c-091b-4011-b1ac-a03b906684f2",
  type: "QUERY",
  ....
 }
]

Retrieving all preferences

The result will be a map where the key is a type and the value is a list.

GET /api/latest/broker/preferences/
 
HTTP/1.1 200 Ok
 
{ "query" : 
 [{
   id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
   type: "QUERY",
   ....
  },

  {
   id: "0e046c3c-091b-4011-b1ac-a03b906684f2",
   type: "QUERY",
   ....
  }
 ],
  "dashboard" : 
  [....]
 }

Create

A single preference can be created with a PUT request against the full URI (one ending including the preferences/type/name).

PUT /api/latest/virtualhost/myvhn/myvh/preferences/query/mypref
{
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 }
}
 
HTTP/1.1 201 Created

One or more preferences can be added to the configured object by sending a POST to the URI preferences/<type> and sending a list, or to the URI /preferences and sending a map. This will add the preferences to those that already exist. No Location header can be returned as multiple URIs will have been potentially created.

(What if one of the new preferences fails a business rule?? Some of the preferences may be added, some not. What is returned?)

POST /api/latest/virtualhost/myvhn/myvh/preferences
{
  "query" : [{
   description: "Acme Hot queues",
   visiblityList: ["ldap.mycompany.org:operators"],
   value: {
     select: "id,name,queueDepthMessages"
     where: "queueDepthMessages > 1000"
   }
  }]
  "dashboard" : [...]
}
 
HTTP/1.1 201 Created

All preferences belonging to a configured object of a given type can be replaced by sending a PUT to the URI preferences/<type> and sending a list, or to the URI /preferences and sending a map. This will replace the preferences that already exist.

When creating a preference, values specified for owner, createdDate, lastUpdatedDate are always ignored by the model. visibilityList will be validated to ensure that domain described by it falls completely within the groups to which the user belongs.

Update

A single preference can be updated with a PUT request against the full URI (one ending including the preferences/type/name). The request is the same as create using PUT above, except the 200 OK will be returned instead of 201 Created.

PUT /api/latest/virtualhost/myvhn/myvh/preferences/query/mypref
{
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 }
}
 
HTTP/1.1 200 Created

One or more preferences can be updated on the configured object by sending a POST to the URI preferences/<type> and sending a list, or to the URI /preferences and sending a map. This will update preferences that already exist the implied locations, and add any that don't.

(What if there are no preference ids on the incoming request? or the ids don't match the preference at the implied location? I think in both case, the request should be rejected. This implies that the caller must call GET to get the id, and return the document with the necessary alter at

POST /api/latest/virtualhost/myvhn/myvh/preferences
{
  "query" : [{
   id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
   description: "Acme Hot queues updated",
   visiblityList: ["ldap.mycompany.org:operators"],
   value: {
     select: "id,name,queueDepthMessages"
     where: "queueDepthMessages > 1000"
   }
  }]
  "dashboard" : [...]
}
 
HTTP/1.1 201 Created

Preferences can be replaced on the configured object by sending a PUT to the URI preferences/<type> and sending a list, or to the URI /preferences and sending a map. This will remove all preferences that exist at that level and replace them with the preferences within the request. It is legal for the client to send an empty list or empty map - this will cause the server to remove the preferences leaving none.

Delete

A single preference can be deleted with a DELETE against its full URI, or ID. All preferences of a particular type, or all preferences belonging to the user can be removed by specifying URI preferences/<type> or /preferences respectively.

The delete will be rejected if the authenticated user does not match the preference's owner, unless the user holds the preference maintainer permission.

DELETE /api/latest/virtualhost/myvhn/myvh/preferences/query/mypref
DELETE /api/latest/virtualhost/myvhn/myvh/preferences/query/?id=75d5d2a4-0573-11e6-b512-3e1d05defe7
DELETE /api/latest/virtualhost/myvhn/myvh/preferences/query

DELETE /api/latest/virtualhost/myvhn/myvh/preferences

REST API /visiblepreferences - retrieve visible preferences for use

This API permits a caller to retrieve all preferences that are visible to the user, excluding those that belong to him.

A client application may permit the user to 'clone' one of these preferences so that the user may make his own modifications. There is no direct backend support for this above a GET to this URI, followed by a suitable POST/PUT to /preferences. The clone is entirely independent of the original.

For GET, /visiblepreferences works in the same manner as /preferences. The other HTTP methods are unsupported.

The a super user, all preferences are visible.

REST API operation /broker/deletePreferences(user)

This broker operation allows the a suitable permissioned user to delete all the preferences belonging to the indicated user. This will support the case where a user has left an organisation.

Preference Storage

In the store an instance of a Preference will look like the example below.

{
 id: <UUID - immutable>,
 name: <name - immutable>,
 type: <type discriminator>
 description: <description>,
 associatedId: <UUID of model object - immutable>,
 owner: <Domain prefixed Principal - normally that belonging to a user - immutable>,
 visibliltyList: [<Domain prefixed Principal - normally that belonging to a group>],
 value: {
   ...
 },
 createdDate: 123456789,
 updatedDate: 123456789
}

Preferences Model Object

At runtime, a Java object will represent the Preference. The interface for this class will look like this. Specialisation of the interface will support the different preference types. The implementations will be backed by a map which will store the name (string) value (object) associations.

(At the moment, I am not imagining analogues of @ManagedAttribute etc).

public interface Preference<V extends PreferenceValue>
{
 UUID getId();
 String getName();
 String getType();
 String getDescription();
 Principal getOwner();
 List<Principal> getVisibiltyList();
 Date getCreatedDate();
 Date getLastUpdatedDate();
 
 V getValue();
 Map<String,Object> getAttributes();
 void setAttributes(Map<String,Object>); 

 void delete();
}

Preference#getAttributes

This will call out to the security manager to authorise the access of the preference.

Returns a r/o map containing the attributes of the preference. Used by the REST API and the Store. The map will have key/value pairs for id, name, type, description, owner, visibilityList, createdDate, lastUpdateDate. It will also have a key value whose value is a map containing the key/value pairs for the preference value itself.

Preference#setAttribute

This will call out to the security manager to authorise the update of the preference. The map will be structured in the same way as that returned by getAttributes.

It then updates the map backing the preference object and invoke the store listener to cause the store to update the preference.

setAttribute must reject any attempt to change the owner. It must also ensure that any the calling user is a member of every group included in the visibilityList.

Preference#delete

This will call out to the security manager to authorise the update of the preference.

It will call back to its owning configured object to have itself removed. The configured object will invoke the store listener to delete the preference.

Configured Object

All ConfiguredObjects will have a set of preferences. This set may be empty. These are the preferences associated with the configured object. The configured object will also have a factory to support the creation of new preferences and finder methods to allow preferences to be looked-up.

Threading....

public interface ConfiguredObject
{
  ...
  Preference createPreference(Map<String,Object>);
  Set<Preference> getPreferences();
  Preference getPreferenceById(UUID);
  Preference getPreferenceByName(String);
  ...
}

ConfiguredObject#createPreference

This will call out to the security manager to authorise the creation of the preference.

It will then use the PreferencesFactory to create the new preference. The factory will be responsible for registering the preference with the configured object to which it is associated.

This method will be responsible for enforcing an cardinality restrictions.

ConfiguredObject#getPreferences()

Returns the preferences associated with the configured object. If the caller is not the system subject, the preferences must be filtered so that only preferences that are visible to the caller are returned in the Set.

The method will call out to the SecurityManager to know if a preference should be returned or not.

ConfiguredObject#getPreferenceBy*()

Returns the preferences associated with the configured object with the given id or name. If the caller is not the system subject, the method must throw an AccessException if the preference is not visible to the caller.

The method will call out to the SecurityManager.

Preferences Factory

There will be a preference factory which will work rather like the object factory. The Model will give the ConfiguredObject its preference factory. BrokerModel will have a hard relationship with a PreferencesFactoryImpl which will know how to lookup specific preference factories and create preference objects. We might automate the creation of preference factories rather like we do for configure objects.

Security Manager

The security manager will gain responsibilities for authorising preference operations.

authoriseCreate(Preference new)
authoriseUpdate(Preference existing, Preference new)
authoriseDelete(Preference)
authoriseAccess(Preference)

For the moment the ordinary user operations we will probably use the existing ACL rules CONFIGURE BROKER and ACCESS VIRTUALHOST.

For the super-user, we will need a new permission will be required (ACCESS PREFERENCES). A user in the identity maintainer role will normally hold this.

Preference Store

We will have a PreferenceStore interface rather like DurableConfigurationStore

open/close
upgrade
visit
create
update
delete

API is in terms of PreferencesRecord (analogous to ConfiguredObjectRecords). The stores themselves will hold the preference records as JSON, but this isn't part of the contract.

AbstractSystemConfig and AbstractVirtualHostNode have a separate preference stores.

The concrete instances of SystemConfig and VirtualHostNode responsible for creating the preference store too.
For storage mechanism BDB, JDBC, Derby there would be a restriction that the preference store must be co-located with the configuration.
For storage mechanism JSON the preferences would be stored in a separate file.

Recovery

At recovery time the AbstractSystemConfig and AbstractVirtualHostNode will need to read all the preferences from their respective stores and create Preference objects. This needs to occur after the configured objects are recovered. In this phase, objects corresponding to the records must be created. The recovery algorithm will be similar to that of configured object recovery as preferences may reference other preferences, so some preferences (Dashboards) may have to await the creation of others before they may be created. As preferences are created they need to be assigned to their associated configured object.

If during the recovery phase, a preference is encountered that refers to a configured object that no longer exists, the preference can be dropped. If a preference is found that refers to a preference that no longer exists, the reference can be removed. The latter would happen if a dashboard of user B refers to a query of user A. If user A deletes their query, user B's reference will be left dangling. On next recovery, the dangling reference must be removed. (The UI should probably just display a ? where the query would have been).

Persistence

The persistent stores need to listen for new preferences, updates to existing preferences or deletes. A listener mechanism similar to ConfigurationChangeListener will be used.

High Level Class Diagram

preferences

Tasks

Sequence of tasks

Implement REST API, preferences model and configured objects changes. After this work it will be possible to view, add and remove preferences at runtime, but preferences will not be persisted. There will be no security.
Change the UI to allow query preferences to be added/removed/viewed
Implement security manager.
Reimplement timezones, refresh period, open tabs(?) as preferences
Implement preference persistence and recovery
Remove old preference model object. Implement a configuration upgrader to remove old preferences from configuration (suggest we do not try and maintain the current user settings)

Once item 1) is done, the remaining steps can be parallelised.

Open Questions

Should preferences have metadata? At the moment I think we can live without it
How will preferences map to AMQP Management?

Child pages

Preference Store

Introduction

High level design

REST API

REST API /preferences - Current user preferences

Retrieve

Retrieving one preference by id

Retrieving one preference by type/name

Retrieving all preferences by type

Retrieving all preferences

Create

Update

Delete

REST API /visiblepreferences - retrieve visible preferences for use

REST API operation /broker/deletePreferences(user)

Preference Storage

Preferences Model Object

Preference#getAttributes

Preference#setAttribute

Preference#delete

Configured Object

ConfiguredObject#createPreference

ConfiguredObject#getPreferences()

ConfiguredObject#getPreferenceBy*()

Preferences Factory

Security Manager

Preference Store

Recovery

High Level Class Diagram

Tasks

Open Questions