THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
AMQP provides the ability to do Point-to-Point, Peer-to-Peer, Pub-Sub, and Eventing. This allows many patterns to be craeted:
Point-to-point
This is one of the simplest use-cases. AMQP allows for this in a few ways.
a.) A client can create a named queue allowing the producer to publish the message to the direct exchange with the key mapping the queue name. This will route the message to that queue.
b.) The above pattern can be extended by specifying a reply-to address in the published messages allowing for the consumer to reply the producer without knowing who it was send from prior to receiving the message.
One-to-many
There are a few patterns that can be used.
...
b.) A 'topic' or 'headers' exchange can also be used. in this case the pattern match is used to send the message to all the bound queues. It can be thought of as a filter allowing you to create just about any One-to-many routing patterns.
Pub-Sub
Topic can be created with the 'topic' or other 'direct' exchange to allow consumer to bind to into the steams of data they care about. This pattern combined with the use of reply-to and Alternate-routing is the staple of what most people use messaging for today.
FAST Reliable Messaging
AMQP 0-10 allows for fully reliable transfers between any two peers. This means that you can publish or subscribe to the broker fully reliable without requiring the need for transactions. This can all be done in async mode with the C++ broker allowing for high throughput while running entirely reliable.
Transactional
AMQP supports two types of transactions in AMQP 0-10, TX and DTX. This allows for local (1PC), and 2PC transaction and the ability to coordinate with a TM (Transaction Manager). The Java broker supports TX, the C++ broker support TX, DTX, XA, JTA for fully ACID transactions. This allows you to commit a single unit of work with may contain enqueues & dequeues either locally on the broker, or in coordination with other transactional resource like RDBMS.
Transient message delivery
By default messages are transient. Transient message can be sent to queues that are durable. They will not be safe stored or recovered, and will perform as any other transient message - fast!
Durable message delivery
There is a header on each message where the message properties are specified, one of these is durability. Messages that are marked as durable and published to a durable queue will be safe stored. Durable messages will survive restart of the broker or cluster.
Federation (Hub-spoke, Trees, graphs)
As AMQP 0-10 is symmetric for peer-to-peer communication all the building block are in place for creating networks of brokers. The C++ broker allows you to link the brokers together using 'qpid-route' and then create routes between the brokers either statically or with dynamic routes.
This allows for a message to be published to one broker and consumed from another broker in the federated broker network. This feature is great to create data-center, or project isolation, but allow cross communication. It also allows networks to be created to scaled. For more details see
And many others, including custom pattern
Message Reply, Rings, Initial Value Caches, Last Value Messaging
All the above cases can be constructed using the AMQP and features of Qpid. For example reply can be constructed using message browsing and setting TTL on the messages. The C++ broker also support ring queues, last value queues, initial value caches on exchanges. With a bit of throught many additional patterns can be constructed.
...
You could force the java client to use kerberos auth by specifying it in the connection URL as follows.
| Code Block |
|---|
amqp://guest:guest@clientid/testpath?brokerlist='tcp://localhost:5672?'&sasl_mechs='GSSAPI'
|
...
| Code Block |
|---|
-Djavax.security.auth.useSubjectCredsOnly=false
# (This will force the SASL GASSPI client to obtain the kerberos credentials explicitly instead of obtaining from the "subject" that owns the currents thread)
-Djava.security.auth.login.config=myjas.conf (this specifies the jass config file)
-Dsun.security.krb5.debug=true (to enable detailed debug info for troubleshooting)
|
...
The C++ broker has plug-ins for Active-Active clustering which keep all the nodes of the cluster in sync. This means that any action that is performed on one of the brokers on the cluster is performed on all of them at the same time. New nodes can be added to the cluster at any time, and removed at any time with no consequences, exept for the extra multi-cast load created for the sync on joining.
How do I start a fault tolerant cluster?
What does the cluster guarantee?
...
The 3rd party persistence store modules may be obtained through anonymous subversion at the following locations:
C++: http://anonsvn.jboss.org/repos/rhmessaging/store/trunk/cpp
Java: http://anonsvn.jboss.org/repos/rhmessaging/store/trunk/java/bdbstore
For further details see 3rd Party Libraries
How do I build the persistence store module from subversion checkouts?
C++
The README file contains detailed instructions, but here is a summary:
- Make sure that both the db4-devel and libaio-devel packages are installed prior to building.
- Make sure that qpid is built and you know the location of the qpid directory (ie the top-level directory containing the python and cpp sub-directories).
- In the store directory, run:
Code Block ./bootstrap ./configure --with-qpid-checkout=/abs/path/to/qpid/dir make
- When built, the store library msgstore.so will be located in the lib/.libs directory.
Java
TODO
How do I use the persistence store module?
C++
- Start the broker making sure that the store module is loaded, ie
Code Block qpidd --load-module=/path/to/msgstore.so --data-dir=/path/to/store-files ...
- Make sure that queues that will handle persistent messages are set durable.
Note title Note: Existing non-persistent queues cannot be made persistent If a queue has been declared without persistence, doing so again with persistence enabled while the old queue still exists in the broker will be ignored. Make sure that when a queue is declared persistent, there is no non-persistent queue of the same name in existence.
- For each message sent to a durable queue, make sure that it is set durable.
Java
TODO
How do I configure the persistence store?
C++
The broker loads help information from each module. To see the help options for the store, load the store module and specify help:
...
The size of the pages in the write page cache is set with the --wcache-page-size option, and sets a size in KiB. (Legal values are powers of 2, ie: 1, 2, 4, 8, 16, 32, 64, 128). Typically small page sizes give improved latency (especially for small messages), but are bad for message throughput, while large page sizes improve throughput but may cause some messages to have higher latencies.
Java
Derby Store
For details of configuring the Derby Store see here
...
For details of using the 3rd party persistent modules see here
| Wiki Markup |
|---|
\[C+\+ store\] What is a RHM_IORES_ENQCAPTHRESH error? |
The journal ran out of space (ENQueue CAPacity THRESHold). The journal is a circular file buffer of fixed capacity set by the journal file size and number of files. When an attempt to write a record causes the journal to exceed an approx. 80% threshold, then the enqueue is rejected with this error code. Dequeues (a written record of a consumed message) may continue, however, as these free up space in the journal. Once space has been freed up, enqueues may continue as normal.
...
- The journal is too small for the size and number of messages being stored. The journal must be made large enough to hold all of the messages you expect to be on the queue at any one moment (a worst-case scenario). Make the journal capacity larger through the use of the --num-jfiles and --jfile-size-pgs parameters.
Info title Rule of thumb for sizing the journal Make the journal twice the size of all the messages you need to store at any one moment in time.
- Messages are not being dequeued (consumed) as expected. Since the store is a circular file buffer, if one un-dequeued (not consumed) message remains, it can eventually "block" the storage of new messages as the buffer gets overwritten.
| Wiki Markup |
|---|
\[C+\+ store\] What is the TPL? What are the \- |
...
-tpl |
...
-\* options for? |
The TPL stands for Transaction Prepared List. The store creates a single instance of a store for storing transaction boundaries called the Transaction Prepared List. Because the TPL is frequently flushed and has very different usage patterns to a normal store, it has been provided with its own set of configuration parameters:
...
| Code Block |
|---|
[commands]$ ./qpid-queue-stats
Traceback (most recent call last):
File "./qpid-queue-stats", line 29, in
from qmf.console import Session, Console
ImportError: No module named qmf.console
|
This problem occurs because the PYTHONPATH environment variable does not include the location of the qpid python files. If you are running from the SVN checkout, add <path>/qpid/python to PYTHONPATH (where <path> is the location of your SVN tree). If you are installing from source, make sure you configure with the same prefix where Python is installed. This is most likely:
...