Broker Management via QMF
As part of the effort to bring more commonality between the Java and C++ brokers I have been attempting to implement QMF management in the Java Broker. Currently the Java Broker can process the majority of the QMF Management commands (although it will not accept agents connecting to it). Ultimatlely the aim must be to allow management and configuration of the brokers using the same set of tools.
In implementing QMF management I have noted a number of questions - some general and related to the nature of QMF, some more specific to the particular set of entities and methods in the management schema for the Qpid (C++) Broker, and some simply relating to the deifnition of individual properties or statistics. I have also noted where I think additions to the existing schema would be useful.
Currently, even if fully implemengted, management through QMF would not be sufficient to duplicate the existing management functions available in the Java Broker (e.g. through the JMX console).
General Comments on the Schema
- Why are there no methods to create entities? Although one can use AMQP commands to accomplish this task it means you have to switch between two distinct command sets to manage the broker; it also means you cannot simply piggyback the QMF management over other transports. (further, for AMQP 1-0 we are removing "management" commands from the core protocol)
GS: See QPID-2317; I'd like using this to be as simple as sending a correctly formatted/encoded message to a special address. I believe with QMFv2 that will indeed be the pattern. It is a useful addition as the new APIs in c++ and python aim to be be protocol independent and thus do not expose the 0-10 declare methods.
- There are no tools for inspecting the contents of messages within the broker. Where a message is stuck on the queue, it is useful to be able to inspect its properties and/or content.
GS: You could do this by browsing the queue (though there is no support then for moving or modifying a message). A scheme that avoids duplicating too much of the standard messaging behaviour would be good.
- Values with High/Low watermarks... the historic low for these will almost invariably be 0, so what is the point?
- Absolute times are specified in nanoseconds... this is "unusual" and it's pretty much impossible to get an accurate absolute time in nanoseconds. Milliseconds for absolute time and nanoseconds for deltas would make more sense.
- The model and methods are not designed to cope with a broker which can support multiple virtual hosts.
- It would be nice to have the ability to configure dynamic event configuration... i.e. the ability to say "publish an event to this address every time this condition is met". You can build this on top of the statistic updates, but it requires you to have a fairly frequent management publish period and a process actively monitoring the output.
General comments on QMF
- have a generic way to be able to create new entities,
- have a way to modify the mutable properties of existing entities.
- when sending schema, where a particular value is of a restricted domain (e.g. has min and/or max values; or takes a value from a restricted enumerated set) then that restricted domain is communicated.
- add ability to "reset" statistic counters... This would actually make low watermark counts useful.
- add ability to query for objects based on criteria (e.g. get me all exchanges whose vhostRef is X)
Comments on Individual Schema Classes
System
Properties
General: what is this supposed to represent - the machine the broker is running on, or the process that is contains the broker?
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
systemId |
uuid |
|
|
Y |
Is this a qpid specific property? |
osName |
sstr |
Operating System Name |
|
Y |
|
nodeName |
sstr |
Node Name |
|
Y |
What exactly is this supposed to be... hostname? |
release |
sstr |
|
|
Y |
What exactly is this supposed to be... release of the OS? |
version |
sstr |
|
|
Y |
What's the difference between this and release? |
machine |
sstr |
|
|
Y |
What is this? |
Broker
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
systemRef |
objId |
System ID |
|
Y |
|
port |
uint16 |
TCP Port for AMQP Service |
|
Y |
A broker may be listening on more than one port - I suggest that we want to have a new entity to represent listening ports/transports |
workerThreads |
uint16 |
Thread pool size |
|
Y |
|
maxConns |
uint16 |
Maximum allowed connections |
|
N |
Java broker currently doesn't support a connection limit |
connBacklog |
uint16 |
Connection backlog limit for listening socket |
|
N |
What is this? |
stagingThreshold |
uint32 |
Broker stages messages over this size to disk |
|
N |
Java Broker does not currently support staging |
mgmtPubInterval |
uint16 |
Interval for management broadcasts |
|
Y |
|
version |
sstr |
Running software version |
|
Y |
|
dataDir |
sstr |
Persistent configuration storage location |
|
Y |
Java Broker currently displays the value of $QPID_HOME here |
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
uptime |
deltaTime |
|
|
Y |
Why have this as a statistic that is going to constantly change? start time as a property would do equally well |
Sggested Additions
- number of current / high watermark of connections
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
( uint32 sequence, lstr body ) echo ( uint32 sequence, lstr body) |
Request a response to test the path to the management broker |
|
Y |
|
void connect ( sstr host, uint32 port, bool durable, sstr authMechanism, sstr username, sstr password, sstr transport) |
Establish a connection to another broker |
|
Y |
Doesn't allow a vhost to be specified, nor is transport defined (is ssl a transport?) |
void queueMoveMessages ( sstr srcQueue, sstr destQueue, uint32 qty) |
Move messages from one queue to another |
|
N |
Queues are local to vhosts, not brokers... since you may have queues with the same name on two separate vhosts on the same broker - this method cannot be implemented on a broker which supports multiple vhosts. Should also add an arguments map to allow filters to be applied |
Suggested Additions
- ability to create virtual hosts
- shutdown/restart broker?
Agent
The Java Broker does not currently support the Agent class
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
connectionRef |
objId |
|
|
N |
|
label |
sstr |
Label for agent |
|
N |
|
registeredTo |
objId |
Broker agent is registered to |
|
N |
|
systemId |
uuid |
Identifier of system where agent resides |
|
N |
|
brokerBank |
uint32 |
Assigned object-id broker bank |
|
N |
|
agentBank |
uint32 |
Assigned object-id agent bank |
|
N |
|
Vhost
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
brokerRef |
objId |
|
|
Y |
|
name |
sstr |
|
|
Y |
|
federationTag |
sstr |
|
|
Y |
|
Statistics
Suggested Additions
- number (and high watermark) of queues/exchanges
- number (and high watermark) of connections
Methods
Suggested Additions
- create queue
- create exchange
- create binding
- delete this vhost
Queue
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
vhostRef |
objId |
|
|
Y |
|
name |
sstr |
|
|
Y |
|
durable |
bool |
|
|
Y |
|
autoDelete |
bool |
|
|
Y |
|
exclusive |
bool |
|
|
Y |
|
arguments |
map |
Arguments supplied in queue.declare |
|
Y |
|
altExchange |
objId |
|
|
Y |
|
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
msgTotalEnqueues |
count64 |
Total messages enqueued |
|
Y |
|
msgTotalDequeues |
count64 |
Total messages dequeued |
|
Y |
|
msgTxnEnqueues |
count64 |
Transactional messages enqueued |
|
|
|
msgTxnDequeues |
count64 |
Transactional messages dequeued |
|
|
|
msgPersistEnqueues |
count64 |
Persistent messages enqueued |
|
Y |
|
msgPersistDequeues |
count64 |
Persistent messages dequeued |
|
Y |
|
msgDepth |
count32 |
Current size of queue in messages |
|
Y |
Is this with or without unacked messages (i.e. does it include all messages for which the dequeue has not yet been committed)? |
byteDepth |
count32 |
Current size of queue in bytes |
|
Y |
|
byteTotalEnqueues |
count64 |
Total messages enqueued |
|
Y |
|
byteTotalDequeues |
count64 |
Total messages dequeued |
|
Y |
|
byteTxnEnqueues |
count64 |
Transactional messages enqueued |
|
|
|
byteTxnDequeues |
count64 |
Transactional messages dequeued |
|
|
|
bytePersistEnqueues |
count64 |
Persistent messages enqueued |
|
Y |
|
bytePersistDequeues |
count64 |
Persistent messages dequeued |
|
Y |
|
consumerCount |
hilo32 |
Current consumers on queue |
|
Y |
|
bindingCount |
hilo32 |
Current bindings |
|
Y |
|
unackedMessages |
hilo32 |
Messages consumed but not yet acked |
|
|
It would seem more useful on a per subscription basis (so we can see which subscriptions are holding on to messages without acking them). |
messageLatency |
mmaTime |
Broker latency through this queue |
|
|
This generates a number of statistics (Max/Min/Avg/Samples) each of which is said to be measured in nanoseconds. What does Samples mean in this context? |
Suggested Additions
- age of oldest message on the queue (and high watermark),
- messages/bytes expired due to TTL
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
void purge ( uint32 request) |
Discard all or some messages on a queue |
|
Y |
Would be nice to add an arguments map so that filters could be supplied. Also making request an IO parameter so you can return how many messages were purged. Finally are the messages supposed to be purged from the head, the tail or randomly throughout the queue? |
Suggested Additions:
- mechanisms to move / copy messages to another queue.
- rename this queue
- delete this queue
- mechanism to inspect messages on the queue
Exchange
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
vhostRef |
objId |
|
|
Y |
|
name |
sstr |
|
|
Y |
|
type |
sstr |
|
|
Y |
|
durable |
bool |
|
|
Y |
|
autoDelete |
bool |
|
|
Y |
|
altExchange |
objId |
|
|
Y |
|
arguments |
map |
Arguments supplied in exchange.declare |
|
|
|
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
producerCount |
hilo32 |
Current producers on exchange |
|
|
How is this supposed to be calculated? |
bindingCount |
hilo32 |
Current bindings |
|
Y |
|
msgReceives |
count64 |
Total messages received |
|
Y |
|
msgDrops |
count64 |
Total messages dropped (no matching key) |
|
Y |
|
msgRoutes |
count64 |
Total routed messages |
|
Y |
|
byteReceives |
count64 |
Total bytes received |
|
Y |
|
byteDrops |
count64 |
Total bytes dropped (no matching key) |
|
Y |
|
byteRoutes |
count64 |
Total routed bytes |
|
Y |
|
Methods
Suggested Additions
- rename
- set AlternateExchange
- delete
Binding
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
exchangeRef |
objId |
|
|
Y |
|
queueRef |
objId |
|
|
Y |
|
bindingKey |
sstr |
|
|
Y |
|
arguments |
map |
|
|
Y |
|
origin |
sstr |
|
|
Y |
definition would be good... why is this not in "arguments" (since that is how it is set)? |
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
msgMatched |
count64 |
|
|
Y |
|
Methods
Suggested Additions
- delete
Subscription
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
sessionRef |
objId |
|
|
Y |
|
queueRef |
objId |
|
|
Y |
|
name |
sstr |
|
|
Y |
|
browsing |
bool |
|
|
Y |
is this supposed to be a pre/non acquire indicator? Is this true only when pre-acquiring and accept-mode is none? |
acknowledged |
bool |
|
|
Y |
is this simply accept-mode = explicit? |
exclusive |
bool |
|
|
Y |
|
creditMode |
sstr |
WINDOW or CREDIT |
|
Y |
|
arguments |
map |
|
|
Y |
|
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
delivered |
count64 |
Messages delivered |
|
Y |
|
Suggested Additions
- available credit
- unacknowledged msg/bytes
- delivered bytes
Methods
Suggested Additions
- delete
Connection
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
vhostRef |
objId |
|
|
Y |
|
address |
sstr |
|
|
Y |
Is this the local or remote address? Whichever, we should also have the other one. |
incoming |
bool |
|
|
Y |
|
SystemConnection |
bool |
Infrastucture/ Inter-system connection (Cluster, Federation, ...) |
|
Y |
|
federationLink |
bool |
Is this a federation link |
|
Y |
|
authIdentity |
sstr |
authId of connection if authentication enabled |
|
Y |
|
remoteProcessName |
sstr |
Name of executable running as remote client |
|
|
These are a subset of the clientProperties sent by the client - should we not just show the client Props? |
remotePid |
uint32 |
Process ID of remote client |
|
|
|
remoteParentPid |
uint32 |
Parent Process ID of remote client |
|
|
|
Suggested Additions
- vhost
- server/client properties maps
- protocol version
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
closing |
bool |
This client is closing by management request |
|
|
Why is this a statistic??? |
framesFromClient |
count64 |
|
|
|
|
framesToClient |
count64 |
|
|
|
|
bytesFromClient |
count64 |
|
|
|
|
bytesToClient |
count64 |
|
|
|
|
Suggested Additions
- attached sessions (with high watermark)
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
void close ( ) |
|
|
|
|
Link
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
vhostRef |
objId |
|
|
Y |
|
host |
sstr |
|
|
Y |
|
port |
uint16 |
|
|
Y |
|
transport |
sstr |
|
|
Y |
What are the valid transports? |
durable |
bool |
|
|
Y |
|
Suggested Additions
- remote vhost
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
state |
sstr |
Operational state of the link |
|
|
Are the valid states defined anywhere? |
lastError |
sstr |
Reason link is not operational |
|
|
|
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
void close ( ) |
|
|
|
|
void bridge ( bool durable, sstr src, sstr dest, sstr key, sstr tag, sstr excludes, bool srcIsQueue, bool srcIsLocal, bool dynamic, uint16 sync) |
Bridge messages over the link |
|
Y |
|
Bridge
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
linkRef |
objId |
|
|
Y |
|
channelId |
uint16 |
|
|
Y |
|
durable |
bool |
|
|
Y |
|
src |
sstr |
|
|
Y |
|
dest |
sstr |
|
|
Y |
|
key |
sstr |
|
|
Y |
|
srcIsQueue |
bool |
|
|
Y |
|
srcIsLocal |
bool |
|
|
Y |
|
tag |
sstr |
|
|
|
what is this and what does it do? |
excludes |
sstr |
|
|
|
what is this and what does it do? |
dynamic |
bool |
|
|
Y |
|
sync |
uint16 |
|
|
|
|
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
void close ( ) |
|
|
|
|
Session
Properties
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
vhostRef |
objId |
|
|
Y |
|
name |
sstr |
|
|
Y |
|
channelId |
uint16 |
|
|
Y |
|
connectionRef |
objId |
|
|
Y |
|
detachedLifespan |
uint32 |
|
|
|
|
attached |
bool |
|
|
Y |
|
expireTime |
absTime |
|
|
|
|
maxClientRate |
uint32 |
|
|
|
|
Suggested Additions
- transactional mode
Statistics
Name |
Type |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|---|
framesOutstanding |
count32 |
|
|
|
|
TxnStarts |
count64 |
Total transactions started |
|
|
Are these supposed to be tx or dtx transactions? or either |
TxnCommits |
count64 |
Total transactions committed |
|
|
|
TxnRejects |
count64 |
Total transactions rejected |
|
|
|
TxnCount |
count32 |
Current pending transactions |
|
|
|
clientCredit |
count32 |
Client message credit |
|
|
|
Suggested Additions:
- subscription count/watermark
- unacked messages/bytes
Methods
Signature |
Description |
C++ |
Java |
Notes |
---|---|---|---|---|
void solicitAck ( ) |
|
|
|
|
void detach ( ) |
|
|
|
|
void resetLifespan ( ) |
|
|
|
|
void close ( ) |
|
|
|
|