THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Running
...
a
...
Qpidd
...
cluster
...
There
...
are
...
several
...
pre-requisites
...
to
...
running
...
a
...
qpidd
...
cluster:
...
Install
...
and
...
configure
...
openais/corosync
...
Qpid
...
clustering
...
uses
...
a
...
multicast
...
protocol
...
provided
...
by
...
the
...
corosync
...
(formerly
...
called
...
openais)
...
library.
...
Install
...
whichever
...
is
...
available
...
on
...
your
...
OS.
...
E.g.
...
in
...
fedora10:
...
yum
...
install
...
corosync.
...
The
...
configuration
...
file
...
is
...
/etc/ais/openais.conf
...
on
...
openais,
...
/etc/corosync.conf
...
on
...
early
...
corosync
...
versions
...
and
...
/etc/corosync/corosync.conf
...
on
...
recent
...
corosync
...
versions.
Depending on the version there may already be a config file installed, or you may need to copy corosync.conf.example to corosync.conf
Here is an example, with places marked that you will
change. ( Below, I will describe how to change the file. )
| Code Block |
|---|
You will need to edit the default file created when you installed Here is an example, with places marked that you will change. ( Below, I will describe how to change the file. ) {code} # Please read the openais.conf.5 manual page totem { version: 2 secauth: off threads: 0 interface { ringnumber: 0 ## You must change this address ## bindnetaddr: 20.0.100.0 mcastaddr: 226.94.32.36 mcastport: 5405 } } logging { debug: off timestamp: on to_file: yes logfile: /tmp/aisexec.log } amf { mode: disabled } {code} |
You
...
must
...
sent
...
the
...
bindnetaddr
...
entry
...
in
...
the
...
configuration
...
file
...
to
...
the
...
network
...
address
...
of
...
your
...
network
...
interface.
...
This
...
must
...
be
...
a
...
real
...
network
...
interface,
...
not
...
the
...
loopback
...
address
...
127.0.0.1
...
You
...
can
...
find
...
your
...
network
...
address by
...
running
...
ifconfig.
...
This
...
will
...
list
...
the
...
host address
...
and
...
the
...
mask,
...
e.g.
| Code Block |
|---|
} inet addr:20.0.20.32 Bcast:20.0.20.255 Mask:255.255.255.0 {code} The bindnetaddr is the logical AND of the inet addr and mask values, in the example above |
The network address for bindnetaddr is the logical AND of the inet addr and mask values, in the example above 20.0.20.0
...
On more recent systems that do not have ifconfig you can use: ip addr
This will give output like
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 24:77:03:42:3a:70 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.104/24 brd 192.168.0.255 scope global dynamic wlp3s0
valid_lft 60405sec preferred_lft 60405sec
inet6 fe80::2677:3ff:fe42:3a70/64 scope link
valid_lft forever preferred_lft forever
The relevant line is inet 192.168.0.104/24 which means that the mask is 24 bits, i.e. 255.255.255.0, so the bindnetaddr should be 192.168.0.
The network address (masked address) is used so that you can use the same configuration for several hosts on the same network. You can also use the complete host address as the bindnetaddr, but in that case each host will need a different configuration file.
Open your firewall
In the above example file, I use mcastport 5405.
This implies that your firewall must allow UDP
protocol over port 5405, or that you disable the firewall
Use the proper identity.
The qpidd process must be started with the correct identity in order to use the corosync/openais library.
For openais and early corosync versions the installation of openAIS/corosync on your system will create a new
group called "ais". The user that starts the qpidd processes of the cluster
must have "ais" as its effective group id. You can create a user specifically for this purpose with ais as the primary group, or
a user that has ais as a secondary group can use "newgrp" to set the primary group to ais when running qpidd.
For more recent corosync 1 versions you no longer need to set your group to "ais" but you do need to create a file in /etc/corosync/uidgid.d/
...
to
...
allow
...
access
...
for
...
whatever
...
user/group
...
ID
...
you
...
want
...
to
...
use.
...
For
...
example
...
create
...
/etc/corosync/uidgid.d/qpid
...
th
...
the
...
contents:
| Code Block |
|---|
uidgid { uid: qpid gid: qpid } h2. Starting a Cluster To be a member of a cluster you must pass the \ |
Finally for corosync 2, you should add the uidgid section shown above directly to your corosync.conf file, not to a separate uidgid file.
Starting a Cluster
Make sure the openais or corosync service has been started.
To be a member of a cluster you must pass the --cluster-name
...
argument
...
to
...
qpidd.
...
This
...
is
...
the
...
only
...
required
...
option
...
to
...
join a cluster,
...
other
...
options
...
can
...
be
...
set
...
as
...
for
...
a
...
normal
...
qpidd.
...
For
...
example
...
to
...
start
...
a
...
cluster
...
of
...
3
...
brokers
...
on
...
the
...
current
...
host
...
Here
...
is
...
an
...
example
...
of
...
starting
...
a
...
cluster
...
of
...
3
...
members,
...
all
...
on
...
the
...
current
...
host
...
but
...
with
...
different
...
ports
...
and
...
different
...
log
...
files:
| Code Block |
|---|
{code} qpidd -p5672 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no qpidd -p5673 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no qpidd -p5674 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no {code} |
In
...
a
...
deployed
...
system,
...
cluster
...
members
...
will
...
normally
...
be
...
on
...
different
...
hosts
...
but
...
for
...
development
...
its
...
useful
...
to
...
be
...
able
...
to
...
create
...
a
...
cluster
...
on
...
a
...
single
...
host.
...
SELinux
...
conflicts
...
Developers
...
will
...
often
...
start
...
openais/corosync
...
as
...
a
...
service
...
like
...
this:
...
service
...
openais
...
start
...
But
...
will
...
then
...
will
...
start
...
a
...
cluster-broker
...
without
...
using
...
the
...
service
...
script
...
like
...
this:
...
/usr/sbin/qpidd
...
--cluster-name
...
my_cluster
...
...
...
If
...
SELinux
...
is
...
in
...
enforcing
...
mode
...
this
...
may
...
cause
...
qpidd
...
to
...
hang
...
due
...
because
...
of
...
the
...
different
...
SELinux
...
contexts.
...
There
...
are
...
3
...
ways
...
to
...
resolve
...
this:
...
- run
...
- both
...
- qpidd
...
- and
...
- openais/corosync
...
- as
...
- services.
...
- run
...
- both
...
- qpidd
...
- and
...
- openais/corosync
...
- as
...
- user
...
- processes.
...
- make
...
- selinux
...
- permissive:
...
To
...
check
...
what
...
mode
...
selinux
...
is
...
running:
| Code Block |
|---|
} # getenforce {code} |
To
...
change
...
the
...
mode:
| Code Block |
|---|
} # setenforce permissive {code} |
Note
...
that
...
in
...
a
...
deployed
...
system
...
both
...
openais/corosync
...
and
...
qpidd
...
should
...
be
...
started
...
as
...
services,
...
in
...
which
...
case
...
there
...
is
...
no
...
problem
...
with
...
SELinux
...
running
...
in
...
enforcing
...
mode.
...
Troubleshooting
...
checklist.
...
If
...
you
...
have
...
trouble
...
starting
...
your
...
cluster,
...
make
...
sure
...
that:
...
- You
...
- have
...
- edited
...
- the
...
- correct
...
- openais/corosync
...
- configuration
...
- file
...
- and
...
- set
...
- bindnetaddr
...
- correctly
...
- Your
...
- firewall
...
- allows
...
- UDP
...
- on
...
- the
...
- openais/corosync
...
- mcastport
...
- Your
...
- effective
...
- group
...
- is
...
- "ais"
...
- (openais/old
...
- corosync)
...
- or
...
- you
...
- have
...
- created
...
- an
...
- appropriate
...
- ID
...
- file
...
- (new
...
- corosync)
...
- Your
...
- firewall
...
- allows
...
- TCP
...
- on
...
- the
...
- ports
...
- used
...
- by
...
- qpidd.
...
- If
...
- you're
...
- starting
...
- openais
...
- as
...
- a
...
- service
...
- but
...
- running
...
- qpidd
...
- directly,
...
- ensure
...
- selinux
...
- is
...
- in
...
- permissive
...
- mode
Troubleshooting FAQ.
- If I got "Daemon startup failed: Cluster-ID mismatch. stores belong to different clusters.", even if I did check name-cluster
- Clean all filesystem database, with --truncate yes
- Delete all file with rm -rf /var/lib/qpidd/* (depending data-dir option)
- If I got "Starting Qpid AMQP daemon: Daemon startup failed: Queue XXXXX: recoverQueues() failed: jexception 0x0c00 jinf::validate() threw JERR_JINF_CVALIDFAIL: Journal compatibility validation failure. (File "/var/lib/qpidd/rhm/jrnl/NNNN/XXXXX/JournalData.jinf": RHM_JDAT_VERSION mismatch: (MessageStoreImpl.cpp:820)"
- Your datastore may be corrupt after a insuffisant disk space error
- Resize your filesystem up
- Clean all filesystem database, with --truncate yes
- Delete all file with rm -rf /var/lib/qpidd/* (depending data-dir option)