You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 23 Next »

Introduction

The Java Broker requires a mechanism that allows a preference, which is owned by a Principal,  to be associated with an arbitrary object with the object hierarchy.  An example of a preference could be:

  • a query
  • a chart
  • a preferred timezone
  • a dashboard layout

A preferences will

  • be owned by a User Principal.
  • associated with exactly one configured object.
  • have a unique ID (UUID)
  • have a visibility which will allow the preference to be private, or shared amongst a group or groups.
  • normally preferences will be updatable/deletable by the Principal that created it but:
    • Special users will be able view/update/delete the preferences of another user.   Users with this permission will normally belong to an identity maintainer role.
    • Deleting a configured object will cause the associated preferences to be deleted automatically.
  • Preferences will be durable (survive broker restart)
  • Some preference types will be understood by the Broker (for example, the query API may later understand a request a execute a query preference).  Other preference types may be private to a client.  The server will be capable of storing/retrieving these private types, but will not accord them server side behaviour.

Preferences associated with a VirtualHost or below need to automatically propagated automatically amongst the nodes within a HA group.  As an operator, this means I will be able to create, say, a query against my HA virtualhost and the same query will be available to me no matter where the mastership resides.

Unlike relationships between CO, preferences are loosely associated with model objects.  This means that an operator will be able to remove, say a queue, even if preferences are still associated with it, be they belong to the operator himself or another Principal. In this case, the preference becomes orphaned. Orphaned preferences will be automatically purged by the system

The UI will allow an operator to clone a copy of a preference of another which is visible to him, in order that he may change his own copy.

Some preferences may refer to other preferences, for example, a dashboard preference may refer to one or more query or chart preferences.

For some preference types, a unlimited number of instances may be associated to a single configured object, for example, QUERY. For other preference types, the cardinality will be zero or one.  An example will be Timezone.

Preferences need to be extensible and allow for the introduction of further preference types  The storage mechanism needs to be versioned so that an automated upgrader can translate an older to a newer one.

By convention, we will associate preferences with either the Broker or Virtualhost.  The model will permit, preferences to be associated anywhere, but we don't anticipate exploiting this at the moment.

High level design

REST API

The Preference API will be exposed over REST and must be part of the versioned API.  To address the preferences the caller will simply append /preferences as the trailing part of the request.  As a consequence, no configured object may have an operation called preferences.

The basic form of the URL to operate on the preferences of the current user only will be:

/api/version/<configured object category>/path/to/configured/object/userpreferences

Users may make preferences visible to other users.  The basic form of the URL to access all the preferences visible to the current user only will be as follows.    The results will not include the preferences that belong to the user.

/api/version/<configured object category>/path/to/configured/object/visiblepreferences

REST API /userpreferences - Current user preferences

Retrieve

GET will retrieve the preference(s) associated with the identified object that belong to the authenticated user only.

The response will be 200 unless:

  • The configured object does not exist (response 404)
  • The preference (by type/name or id) does not exist (response 404)

If no preferences are associated with the configured object the response will be an empty JSON list.

Retrieving one preference by id

 The result will be exactly one object.

GET /api/latest/broker/userpreferences?id=75d5d2a4-0573-11e6-b512-3e1d05defe78
HTTP/1.1 200 Ok
 
{
 id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
 type: "QUERY",
 name: "mypref",
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 },
 owner: "ldap.mycompany.org:kwall"
 createdDate: 123456789,
 updatedDate: 123456789
}
Retrieving one preference by type/name

The result will be exactly one object

GET /api/latest/broker/userpreferences/query/mypref
HTTP/1.1 200 Ok
 
{
 id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
 type: "QUERY",
 ....
}
Retrieving all preferences by type

The result will be a list of objects.

GET /api/latest/broker/userpreferences/query
HTTP/1.1 200 Ok
 
[{
  id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
  type: "QUERY",
  ....
 },
 {
  id: "0e046c3c-091b-4011-b1ac-a03b906684f2",
  type: "QUERY",
  ....
 }
]
Retrieving all preferences

The result will be a map where the key is a type and the value is a list.

GET /api/latest/broker/userpreferences/
 
HTTP/1.1 200 Ok
 
{ "query" : 
 [{
   id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
   type: "QUERY",
   ....
  },

  {
   id: "0e046c3c-091b-4011-b1ac-a03b906684f2",
   type: "QUERY",
   ....
  }
 ],
  "dashboard" : 
  [....]
 }


Create

A single preference can be created with a PUT request against the full URI (one ending including the preferences/type/name).

PUT /api/latest/virtualhost/myvhn/myvh/userpreferences/query/mypref
{
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 }
}
 
HTTP/1.1 201 Created

One or more preferences can be added to the configured object by sending a POST to the URI userpreferences/<type> and sending a list, or to the URI /userpreferences and sending a map.   This will add the preferences to those that already exist.  No Location header can be returned as multiple URIs will have been potentially created.

(What if one of the new preferences fails a business rule??  Some of the preferences may be added, some not. What is returned?)

POST /api/latest/virtualhost/myvhn/myvh/userpreferences
{
  "query" : [{
   description: "Acme Hot queues",
   visiblityList: ["ldap.mycompany.org:operators"],
   value: {
     select: "id,name,queueDepthMessages"
     where: "queueDepthMessages > 1000"
   }
  }]
  "dashboard" : [...]
}
 
HTTP/1.1 201 Created

All preferences belonging to a configured object of a given type can be replaced by sending a PUT to the URI userpreferences/<type> and sending a list, or to the URI /userpreferences and sending a map.   This will replace the preferences that already exist. 

When creating a preference, values specified for owner, createdDate, lastUpdatedDate are always ignored by the model.  visibilityList will be validated to ensure that domain described by it falls completely within the groups to which the user belongs.

Update

A single preference can be updated with a PUT request against the full URI (one ending including the /userpreferences/type/name).   The request is the same as create using PUT above, except the 200 OK will be returned instead of 201 Created.

PUT /api/latest/virtualhost/myvhn/myvh/userpreferences/query/mypref
{
 description: "Acme Hot queues",
 visiblityList: ["ldap.mycompany.org:operators"],
 value: {
   select: "id,name,queueDepthMessages"
   where: "queueDepthMessages > 1000"
 }
}
 
HTTP/1.1 200 Created

One or more preferences can be updated on the configured object by sending a POST to the URI userpreferences/<type> and sending a list, or to the URI /userpreferences and sending a map.   This will update preferences that already exist the implied locations, and add any that don't.

(What if there are no preference ids on the incoming request? or the ids don't match the preference at the implied location? I think in both case, the request should be rejected.   This implies that the caller must call GET to get the id, and return the document with the necessary alter at

POST /api/latest/virtualhost/myvhn/myvh/userpreferences
{
  "query" : [{
   id: "75d5d2a4-0573-11e6-b512-3e1d05defe7",
   description: "Acme Hot queues updated",
   visiblityList: ["ldap.mycompany.org:operators"],
   value: {
     select: "id,name,queueDepthMessages"
     where: "queueDepthMessages > 1000"
   }
  }]
  "dashboard" : [...]
}
 
HTTP/1.1 201 Created

Preferences can be replaced on the configured object by sending a PUT to the URI preferences/<type> and sending a list, or to the URI /preferences and sending a map.  This will remove all preferences  that exist at that level and replace them with the preferences within the request.  It is legal for the client to send an empty list or empty map - this will cause the server to remove the preferences leaving none.

Delete

A single preference can be deleted with a DELETE against its full URI, or ID.  All preferences of a particular type, or all preferences belonging to the user can be removed by specifying URI /userpreferences/<type> or /userpreferences respectively.

The delete will be rejected if the authenticated user does not match the preference's owner, unless the user holds the preference maintainer permission.

DELETE /api/latest/virtualhost/myvhn/myvh/userpreferences/query/mypref
DELETE /api/latest/virtualhost/myvhn/myvh/userpreferences/query/?id=75d5d2a4-0573-11e6-b512-3e1d05defe7
DELETE /api/latest/virtualhost/myvhn/myvh/userpreferences/query

DELETE /api/latest/virtualhost/myvhn/myvh/preferences

REST API /visiblepreferences - retrieve visible preferences for use

This API permits a caller to retrieve all preferences that are visible to the user, excluding those that belong to him.

A client application may permit the user to 'clone' one of these preferences so that the user may make his own modifications.  There is no direct backend support for this above a GET to this URI, followed by a suitable POST/PUT to /preferences.  The clone is entirely independent of the original.

For GET, /visiblepreferences works in the same manner as /userpreferences.   The other HTTP methods are unsupported.

The a super user, all preferences are visible.

REST API operation /broker/deletePreferences(user)

This broker-level operation allows the a suitable permissioned user to delete all the preferences belonging to the indicated user.  This will support the case where a user has left an organisation.

Configured Object Changes

All ConfiguredObjects will have a set of preferences representing the set of preferences associated with the configured object.

The ConfiguredObject is responsible for:

  • creation of new preference(s)
  • update of existing preference(s)
  • delete of existing preferences(s)
  • access of some/all of the preference(s)
  • makes security decisions by talking to the SecurityManager.

The ConfiguredObject will use a dedicated single thread for the creation/deletion/update of preferences. This approach will allow the individual REST API interactions to be atomic even when the request embodies multiple preferences.  It is likely that the preference executor thread will be shared by all configured objects within a contiguous part of the model - that is, there will be a broker preference executor, and a per virtualhost preference executor.

The REST API interacts with preferences via the ConfiguredObject API.

public interface ConfiguredObject
{
  ...
  List<Preference> createPreferences(List<Map<String,Object>>); 
  Set<Preference> getPreferences(PreferenceFilter);
 
  void updatePreferences(List<Map<String,Object>>);
  
  Set<Preference> deletePreferences(PreferenceFilter);
  ...
}

 

ConfiguredObject#createPreferences

This method will create one or more preferences as an atomic operation.  This method first will:

  • establish that the business rules (name uniqueness, cardinality rules) are not broken by the proposed new preferences
  • check with the security manager that the creation of the preferences is allowed

It will then:

  • use the PreferencesFactory to create the new preference(s) and put them in a local preference registry
  • tell the store listener that new preferences have been created.

As preferences are mutated by a single thread and the business rules are checked for violations up front, the need to 'rollback' successfully created preferences if a later one fails is avoided.

ConfiguredObject#updatePreferences

This method will update one or more preferences as an atomic operation.  This method first will:

  • establish that the business rules (name uniqueness, cardinality rules) are not broken by the proposed changes
  • check with the security manager that the update of the preferences is allowed

It will then:

  • use the PreferencesFactory to create replacement preference object(s)
  • tell the store listener that preferences have been updated.

As preferences are mutated by a single thread and the business rules are checked for violations up front, the need to 'rollback' successfully created preferences if a later one fails is avoided.

ConfiguredObject#getPreferences()

This method gets the preferences that match a caller supplied filter and are accessible to the authenticated user.

The caller (typically the REST layer) will pass a filter with predicates:

  • on type, type/name or id
  • if a /userpreferences request - a predicate that encapsulates preference.owner == authenticate user
    if a /visiblepreferences request - a predicate that encapsulate authenticated user is member of a group included in preference.visiblityList && preference.owner <> authenticate user

This method will call out to the SecurityManager is accessible.  Only those passing this check will be included in the Set.

ConfiguredObject#deletePreferences()

This method deletes the preferences that match a caller supplied filter and belong to the authenticated user.  The caller (typically the REST layer) will passing a filter which filters on type, type/name or id.

This method will call out to the SecurityManager to determine if the preference may be deleted.

This method will return the preferences that were deleted, allowing a future version of the REST API to return the delete content as part of the DELETE response.

Preferences Model Object

Preference object represents an instance of a preference.      Each Preference will have a PreferenceValue encapsulating the type specific details of preference.  There will be a GenericPreferenceValue used to represent arbitrary preferences that the server requires no knowledge (these will have a type name begining X-).  There will also specific concrete implementations of PreferenceValue that represent preferences types which the server has knowledge.

Concrete instances of Preference/PreferenceValue will be immutable.

Preference Objects do not make access decisions.

public interface Preference<V extends PreferenceValue>
{
 UUID getId();
 String getName();
 String getType();
 String getDescription();
 Principal getOwner();
 List<Principal> getVisibiltyList();
 Date getCreatedDate();
 Date getLastUpdatedDate();
 
 V getValue();
 Map<String,Object> getAttributes();
}
 
public interface PreferenceValue
{
  Map<String,Object> getAttributes();
}

Preference#getAttributes

Returns a r/o map containing the attributes of the preference.  The map will have key/value pairs for id, name, type, description, owner, visibilityList, createdDate, lastUpdateDate.  It will also have a key value whose value is a map containing the key/value pairs for the preference value itself.

Preferences Factory

There will be a preference factory which will work rather like the object factory.  The Model will give the ConfiguredObject a preference factory.  BrokerModel will have a hard relationship with a PreferencesFactoryImpl which will know how to create Preference and PreferenceValue objects from the attributes.

The factory will use the type discriminator to determine the actual type of PreferenceValue.  If the discriminator begins with "X-" the GenericPreferenceValue will be used, other a specific concrete PreferenceValue will be used.

Preference Storage

In the store an instance of a Preference will look like the example below. 

{
 id: <UUID - immutable>,
 name: <name - immutable>,
 type: <type discriminator>
 description: <description>,
 associatedId: <UUID of model object - immutable>,
 owner: <Domain prefixed Principal - normally that belonging to a user - immutable>,
 visibliltyList: [<Domain prefixed Principal - normally that belonging to a group>],
 value: {
   ...
 },
 createdDate: 123456789,
 updatedDate: 123456789
} 

 

Security Manager

The security manager will gain responsibilities for authorising preference operations.

  • authoriseCreate(Preference new)
  • authoriseUpdate(Preference existing, Preference new)
  • authoriseDelete(Preference)
  • authoriseAccess(Preference)

For the moment the ordinary user operations we will probably use the existing ACL rules CONFIGURE BROKER and ACCESS VIRTUALHOST.

For the super-user, we will need a new permission will be required (ACCESS PREFERENCES).  A user in the identity maintainer role will normally hold this.

Preference Store

We will have a PreferenceStore interface rather like DurableConfigurationStore

  • open/close

  • upgrade
  • visit
  • create
  • update
  • delete

API is in terms of PreferencesRecord (analogous to ConfiguredObjectRecords).  The stores themselves will hold the preference records as JSON, but this isn't part of the contract.

AbstractSystemConfig and AbstractVirtualHostNode have a separate preference stores.

  • The concrete instances of SystemConfig and VirtualHostNode responsible for creating the preference store too.
  • For storage mechanism BDB, JDBC, Derby there would be a restriction that the preference store must be co-located with the configuration.
  • For storage mechanism JSON the preferences would be stored in a separate file.

Recovery

At recovery time the AbstractSystemConfig and AbstractVirtualHostNode will need to read all the preferences from their respective stores and create Preference objects.    This needs to occur after the configured objects are recovered. In this phase, objects corresponding to the records must be created.  The recovery algorithm will be similar to that of configured object recovery as preferences may reference other preferences, so some preferences (Dashboards) may have to await the creation of others before they may be created.   As preferences are created they need to be assigned to their associated configured object.

If during the recovery phase, a preference is encountered that refers to a configured object that no longer exists, the preference can be dropped.  If a preference is found that refers to a preference that no longer exists, the reference can be removed.  The latter would happen if a dashboard of user B refers to a query of user A.  If user A deletes their query, user B's reference will be left dangling.  On next recovery, the dangling reference must be removed.  (The UI should probably just display a ? where the query would have been).

Persistence

The persistent stores need to listen for new preferences, updates to existing preferences or deletes.  A listener mechanism similar to ConfigurationChangeListener will be used.

High Level Class Diagram

 

preferences

 

Tasks

Sequence of tasks

  1. Implement REST API, preferences model and configured objects changes. After this work it will be possible to view, add and remove preferences at runtime, but preferences will not be persisted.  There will be no security.
  2. Change the UI to allow query preferences to be added/removed/viewed
  3. Implement security manager.
  4. Reimplement timezones, refresh period, open tabs(?) as preferences
  5. Implement preference persistence and recovery
  6. Remove old preference model object.  Implement a configuration upgrader to remove old preferences from configuration (suggest we do not try and maintain the current user settings)

Once item 1) is done, the remaining steps can be parallelised.

Open Questions

  1. Should preferences have metadata?  At the moment I think we can live without it
  2. How will preferences map to AMQP Management?
  • No labels