Operational Logging - Status Update - Test Specification

Test Specification

Overview

The Test Specification will detail a 1:1 mapping from specification to test case.
Each test specification in the list will include:

• Functional description of what is being tested.
• Input(actions and/or data)
• Expected outputs:
  • ... that will cause failure
  • ... that can safely be ignored.

These details will then be used as the basis of each test that is created allowing for better maintainability in the test code.

Operational Test Cases

This section enumerates the various operational tests described in the Test Plan identified from the Functional Specification. This text should form the basis of the Technial Documenation for the specified test class.

Broker Test Suite

The Broker test suite validates that the follow log messages as specified in the Functional Specification.

BRK-1001 : Startup : Version: <Version> Build: <Build>
BRK-1002 : Starting : Listening on <Transport> port <Port>
BRK-1003 : Shuting down : <Transport> port <Port>
BRK-1004 : Ready
BRK-1005 : Stopped
BRK-1006 : Using configuration : <path>
BRK-1007 : Using logging configuration : <path>

These messages should only occur during startup. The tests need to verify the order of messages. In the case of the BRK-1002 and BRK-1003 the respective ports should only be available between the two log messages.

Broker Startup

testBrokerStartupConfiguration

Description: On startup the broker must report the active configuration file. The logging system must output this so that we can know what configuration is being used for this broker instance.
Input:
The value of -c specified on the command line.
Output:

<date> MESSAGE BRK-1006 : Using configuration : <config file>

Constraints:
This MUST BE the first BRK log message.
Validation Steps:

1. This is first BRK log message.
2. The BRK ID is correct
3. The config file is the full path to the file specified on the commandline.

testBrokerStartupCustomLog4j

Description:
On startup the broker must report correctly report the log4j file in use. This is important as it can help diagnose why logging messages are not being reported. The broker must also be capable of correctly recognising the command line property to specify the custom logging configuration.
Input:
The value of -l specified on the command line.
Output:

<date> MESSAGE BRK-1007 : Using logging configuration : <log4j file>

Validation Steps:

1. The BRK ID is correct
2. This should occur before the BRK-1001 : Startup message
3. The log4j file is the full path to the file specified on the commandline.

testBrokerStartupDefaultLog4j

Description:
On startup the broker must report correctly report the log4j file in use. This is important as it can help diagnose why logging messages are not being reported.
Input:
No custom -l value should be provided on the command line so that the default value is correctly reported.
Output:

<date> MESSAGE BRK-1007 : Using logging configuration : <$QPID_HOME>/etc/log4j.xml

Validation Steps:

1. The BRK ID is correct
2. This occurs before the BRK-1001 startup message.
3. The log4j file is the full path to the file specified on the commandline.

testBrokerStartupStartup

Description: On startup the broker reports the broker version number and svn build revision. This information is retrieved from the resouce 'qpidversion.properties' which is located via the classloader.
Input: The 'qpidversion.properties' file located on the classpath.
Output:

<date> MESSAGE BRK-1001 : Startup : qpid Version: 0.6 Build: 767150

Validation Steps:

1. The BRK ID is correct
2. This occurs before any BRK-1002 listenting messages are reported.

testBrokerStartupListeningTCPDefault

Description:
On startup the broker may listen on a number of ports and protocols. Each of these must be reported as they are made available.
Input:
The default configuration with no SSL
Output:

<date> MESSAGE BRK-1002 : Starting : Listening on TCP port 5672

Constraints:
Additional broker configuration will occur between the Startup(BRK-1001) and Starting(BRK-1002) messages depending on what VirtualHosts are configured.
Validation Steps:

1. The BRK ID is correct
2. This occurs after the BRK-1001 startup message
3. Using the default configuration a single BRK-1002 will be printed showing values TCP / 5672

testBrokerStartupListeningTCPSSL

Description:
On startup the broker may listen on a number of ports and protocols. Each of these must be reported as they are made available.
Input:
The default configuration with SSL enabled
Output:

<date> MESSAGE BRK-1002 : Starting : Listening on TCP port 5672
<date> MESSAGE BRK-1002 : Starting : Listening on TCP/SSL port 8672

Constraints:
Additional broker configuration will occur between the Startup(BRK-1001) and Starting(BRK-1002) messages depending on what VirtualHosts are configured.
Validation Steps:

1. The BRK ID is correct
2. This occurs after the BRK-1001 startup message
3. With SSL enabled in the configuration two BRK-1002 will be printed (order is not specified)
   1. One showing values TCP / 5672
   2. One showing values TCP/SSL / 5672

testBrokerStartupReady

Description:
The final messasge the broker will print when it has performed all initialisation and listener startups will be to log the BRK-1004 Ready message
Input:
No input, all succesful broker startups will show BRK-1004 messages.
Output:

2009-07-09 15:50:20 +0100 MESSAGE BRK-1004 : Ready

Validation Steps:

1. The BRK ID is correct
2. This occurs after the BRK-1001 startup message
3. This must be the last message the broker prints after startup. Currently, if there is no further interaction with the broker then there should be no more logging.

Broker Shutdown

testBrokerShutdownListeningTCPDefault

Description:
On startup the broker may listen on a number of ports and protocols. Each of these must then report a shutting down message as they stop listening.
Input:
The default configuration with no SSL
Output:

<date> MESSAGE BRK-1003 : Shutting down : TCP port 5672

Validation Steps:

1. The BRK ID is correct
2. Only TCP is reported with the default configuration with no SSL.
3. The default port is correct
4. The port is not accessible after this message

testBrokerShutdownListeningTCPSSL

Description:
On startup the broker may listen on a number of ports and protocols. Each of these must then report a shutting down message as they stop listening.
Input:
The default configuration with SSL enabled
Output:

<date> MESSAGE BRK-1003 : Shutting down : TCP port 5672
<date> MESSAGE BRK-1003 : Shutting down : TCP/SSL port 8672

Validation Steps:

1. The BRK ID is correct
2. With SSL enabled in the configuration two BRK-1003 will be printed (order is not specified)
3. The default port is correct
4. The port is not accessible after this message

testBrokerShutdownStopped

Description:
Input:
No input, all clean broker shutdowns will show BRK-1005 messages.
Output:

<date> MESSAGE BRK-1005 : Stopped

Constraints:
This is the LAST message the broker will log.
Validation Steps:

1. The BRK ID is correct
2. This is the last message the broker will log.

Management Console Test Suite

The Management Console test suite validates that the follow log messages as specified in the Functional Specification.

This suite of tests validate that the management console messages occur in correctly and according to the following format:

MNG-1001 : Startup
MNG-1002 : Starting : <service> : Listening on port <Port>
MNG-1003 : Shuting down : <service> : port <Port>
MNG-1004 : Ready
MNG-1005 : Stopped
MNG-1006 : Using SSL Keystore : <path>

Management Startup

testManagementStartupEnabled

Description:
Using the startup configuration validate that the managment startup messasge is logged correctly.
Input:
Standard configuration with management enabled
Output:

<date> MNG-1001 : Startup

Constraints:
This is the FIRST message logged by MNG
Validation Steps:

1. The BRK ID is correct
2. This is the FIRST message logged by MNG

testManagementStartupDisabled

Description:
Verify that when management is disabled in the configuration file the startup message is not logged.
Input:
Standard configuration with management disabled
Output:
NO MNG messages
Validation Steps:

1. Validate that no MNG messages are produced.

testManagementStartupRMIRegistry

Description:
Using the default configuration validate that the RMI Registry socket is correctly reported as being opened
Input:
The default configuration file
Output:

<date> MESSAGE MNG-1002 : Starting : RMI Registgry : Listening on port 8999

Constraints:
The RMI ConnectorServer and Registry log messages do not have a prescribed order
Validation Steps:

1. The MNG ID is correct
2. The specified port is the correct '8999'

testManagementStartupRMIRegistryCustom

Description:
Using the default configuration validate that the RMI Registry socket is correctly reported when overridden via the command line.
Input:
The default configuration file and a custom -m value
Output:

<date> MESSAGE MNG-1002 : Starting : RMI Registgry : Listening on port <port>

Constraints:
The RMI ConnectorServer and Registry log messages do not have a prescribed order
Validation Steps:

1. The MNG ID is correct
2. The specified port is as specified on the commandline.

testManagementStartupRMIConnectorServer

Description:
Using the default configuration validate that the RMI ConnectorServer socket is correctly reported as being opened
Input:
The default configuration file
Output:

<date> MESSAGE MNG-1002 : Starting : RMI ConnectorServer : Listening on port 9099

Constraints:
The RMI ConnectorServer and Registry log messages do not have a prescribed order
Validation Steps:

1. The MNG ID is correct
2. The specified port is the correct '9099'

testManagementStartupRMIConnectorServerCustom

Description:
Using the default configuration validate that the RMI Registry socket is correctly reported when overridden via the command line.
Input:
The default configuration file and a custom -m value
Output:

<date> MESSAGE MNG-1002 : Starting : RMI ConnectorServer : Listening on port <port>

Constraints:
The RMI ConnectorServer and Registry log messages do not have a prescribed order
Validation Steps:

1. The MNG ID is correct
2. The specified port is as specified on the commandline.

testManagementStartupSSLKeystore

Description:
Using the default configuration with SSL enabled for the management port the SSL Keystore path should be reported via MNG-1006
Input:
Management SSL enabled default configuration.
Output:

<date> MESSAGE MNG-1006 : Using SSL Keystore : test_resources/ssl/keystore.jks

Validation Steps:

1. The MNG ID is correct
2. The keystore path is as specified in the configuration

testManagementStartupReady

Description:
Using the default configuration the final stage of management startup is to report a MNG-1004 Ready message.
Input:
Default broker configuration.
Output:

<date> MESSAGE MNG-1004 : Ready

Validation Steps:

1. The MNG ID is correct
2. There has been a MNG-1001 message
3. There has been at least one MNG-1002 Listenting message
4. No further MNG messages are produced as part of the startup process, i.e. before broker use.

Management Shutdown

testManagementShutdownRMIRegistry

Description:
Using the default configuration the management RMI Registry will start and so on shutdown it will log that it is shutting down.
Input:
The default configuration file.
Output:

<date> MNG-1003 : Shutting down : RMI Registgry : Listening on port 8999

Validation Steps:

1. The MNG ID is correct
2. The MNG-1004 message has been logged.

testManagementShutdownRMIConnectorServer

Description:
Using the default configuration the management RMI ConnectorServer will start and so on shutdown it will log that it is shutting down.
Input:
The default configuration file.
Output:

MNG-1003 : Shutting down : RMI ConnectorServer : Listening on port 9099

Validation Steps:

1. The MNG ID is correct
2. The MNG-1004 message has been logged.

testManagementShutdownStopped

Description:
On final shutdown the management console will report that it has stopped. All MNG logging must be complete before this message is logged.
Input:
The default configuration file.
Output:

MNG-1005 : Stopped

Validation Steps:

1. The MNG ID is correct
2. The MNG-1004 message has been logged.
3. For each MNG-1002 message that was logged a MNG-1003 is also logged before this message.
4. This is the last MNG message reported.

Performance Test Cases

TBC