Introduction

The qpid-broker-j sources bundles a performance test frame work. The main goal is to determine peak sustainable throughput numbers. That is the number of messages (of a certain size) per second that can pass through the broker for a long period of time. It explicitly does not test peak throughput and/or latency.

Throughout these instructions whenever you encounter an environment variable syntax like ${host} you should replace that with whatever is appropriate to your use case.

Code Performance vs Environment Performance

When setting up and running the performance tests one has to keep in mind what it is one wants to test. The performance of the code itself or the performance of the code in a certain environment. While it is not possible to test the code in isolation independent of the environment one can undertake certain steps to minimize the impact of the environment.

Things in the environment that influence the outcome of the performance tests:

• Memory. Performance tests might need a lot of memory.

...

• When running out of memeory the broker will flow messages to disk and the JVM will start garbage collecting. Ensure that the JVM has sufficient heap and direct memory (-Xmx and -XX:MaxDirectMemorySize respectively).
• CPU. As few as possible other processes should be running on the machines involved in the test.
• Bandwidth. If the broker and clients communicate over the network the bandwidth can become a limiting factor. Especially transient messages are known to saturate a 100 MBit connection.
• JVM. Different JVMs will have different performance characteristics. Furthermore things like JIT optimizations might have a warm-up phase which can influence the performance outcome (increasing test duration to some extent remedies this).
• Test duration number of runs. Longer and more runs typically result in better and more stable/reliable results. This is a trade of between result quality and time it takes to run the performance tests.
• Test setup. Especially the number of producer/consumer connections has an impact on the performance outcome.

Running the Performance Tests from the Command Line

1. Start the broker on ${broker_host} (default: localhost)
2. ensure the broker has http management enabled and listening on port ${broker_port_http} (default: 8080)
3. Configure the broker to listen on AMQP port ${broker_port} (default: 5671)
4. Configure the broker to accept the credentials ${username} and ${password} (default: admin:admin)
5. Configure the broker's http plugin to accept basic authentication
6. If needed configure a KeyStore and a AMQPS port at ${broker_ssl_port}
7. Invoke maven to run the performance tests

   1. Invoke maven on the command line:

      Code Block
      language text
      mvn dependency:tree integration-test -f ${path_to_qpid_sources}/perftests/pom.xml -DskipTests -Dperftests=qpid-jms-client -Dperftests.management-url=http://${broker_host}:${broker_port_http} -Dpertests.management-user=${username} -Dperftests.management-password=${password} -Dperftests.messaging-hostport-plain=${broker_port} -Dperftests.messaging-hostport-plain=${broker_ssl_port} -Dperftests.messaging-user=${username} -Dperftests.messaging-password=${password} -Dperftests.hillclimb=true

   2. if all the defaults are used this can be simplified to

      Code Block
      language text
      mvn dependency:tree integration-test -f ${path_to_qpid_sources}/perftests/pom.xml -DskipTests -Dperftests=qpid-jms-client -Dperftests.hillclimb=true

   3. Other interesting options are as follows:
      1. perftests.hillclimb.minimum_delta
      2. perftests.hillclimb.max_runs
      3. qpid.disttest.duration
8. The results will end up in the current working directory. The result summary will be in test-summary.csv while individual results will be written to defaultTests.csv.