THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Description of testcases in Kafka system test
Replication Basic |
|
|
|
|
|
|---|---|---|---|---|---|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0001 | No. of topic : 1 | None | Sync | -1 | 0 |
testcase_0002 | No. of topic : 1 | None | Sync | -1 | 1 |
testcase_0003 | No. of topic : 1 | None | Sync | 1 | 1 |
testcase_0004 | No. of topic : 1 | None | Async | -1 | 1 |
testcase_0005 | No. of topic : 1 | None | Async | 1 | 1 |
testcase_0006 | No. of topic : 1 | None | Sync | -1 | 1 |
testcase_0007 | No. of topic : 1 | None | Async | -1 | 1 |
testcase_0008 | No. of topic : 1 | None | Sync | 1 | 1 |
testcase_0009 | No. of topic : 1 | None | Async | 1 | 1 |
testcase_0010 | No. of topic : 1 | None | Async | 1 | 1 |
testcase_0011 | No. of topic : 1 | None | Async | 1 | 1 |
Replication Basic |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0021 | No. of topic : 2 | None | Sync | -1 | 0 |
testcase_0022 | No. of topic : 2 | None | Sync | 1 | 0 |
testcase_0023 | No. of topic : 2 | None | Async | 1 | 1 |
testcase_0024 | No. of topic : 2 | None | Async | 1 | 1 |
Replication with Leader Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0101 | No. of topic : 1 | kill -15 | Sync | -1 | 0 |
testcase_0102 | No. of topic : 1 | kill -15 | Sync | -1 | 1 |
testcase_0103 | No. of topic : 1 | kill -15 | Sync | 1 | 1 |
testcase_0104 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
testcase_0105 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
testcase_0106 | No. of topic : 1 | kill -15 | Sync | -1 | 1 |
testcase_0107 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
testcase_0108 | No. of topic : 1 | kill -15 | Sync | 1 | 1 |
testcase_0109 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
testcase_0110 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
Replication with Leader Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0111 | No. of topic : 1 | kill -15 | Sync | -1 | 0 |
testcase_0112 | No. of topic : 1 | kill -15 | Async | -1 | 0 |
testcase_0113 | No. of topic : 1 | kill -15 | Sync | 1 | 0 |
testcase_0114 | No. of topic : 1 | kill -15 | Async | 1 | 0 |
testcase_0115 | No. of topic : 1 | kill -15 | Sync | -1 | 1 |
testcase_0116 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
testcase_0117 | No. of topic : 1 | kill -15 | Sync | 1 | 1 |
testcase_0118 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
testcase_0119 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
Replication with Leader Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0121 | No. of topic : 2 | kill -15 | Sync | -1 | 0 |
testcase_0122 | No. of topic : 2 | kill -15 | Sync | 1 | 0 |
testcase_0123 | No. of topic : 2 | kill -15 | Async | 1 | 1 |
testcase_0124 | No. of topic : 2 | kill -15 | Sync | -1 | 0 |
testcase_0125 | No. of topic : 2 | kill -15 | Sync | 1 | 0 |
testcase_0126 | No. of topic : 2 | kill -15 | Sync | -1 | 1 |
testcase_0127 | No. of topic : 2 | kill -15 | Sync | 1 | 1 |
testcase_0128 | No. of topic : 2 | kill -15 | Async | -1 | 1 |
Replication with Leader Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0131 | No. of topic : 2 | kill -15 | Sync | -1 | 0 |
testcase_0132 | No. of topic : 2 | kill -15 | Sync | 1 | 0 |
testcase_0133 | No. of topic : 2 | kill -15 | Async | 1 | 1 |
testcase_0134 | No. of topic : 2 | kill -15 | Async | -1 | 1 |
Replication with Leader Hard Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0151 | No. of topic : 1 | kill -9 | Sync | -1 | 0 |
testcase_0152 | No. of topic : 1 | kill -9 | Async | -1 | 0 |
testcase_0153 | No. of topic : 1 | kill -9 | Sync | 1 | 0 |
testcase_0154 | No. of topic : 1 | kill -9 | Async | 1 | 0 |
testcase_0155 | No. of topic : 1 | kill -9 | Sync | -1 | 1 |
testcase_0156 | No. of topic : 1 | kill -9 | Async | -1 | 1 |
testcase_0157 | No. of topic : 1 | kill -9 | Sync | 1 | 1 |
testcase_0158 | No. of topic : 1 | kill -9 | Async | 1 | 1 |
testcase_159 | No. of topic : 1 | kill -9 | Async | -1 | 1 |
Replication with Controller Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0201 | No. of topic : 1 | kill -15 | Sync | -1 | 0 |
testcase_0202 | No. of topic : 1 | kill -15 | Async | -1 | 0 |
testcase_0203 | No. of topic : 1 | kill -15 | Sync | 1 | 0 |
testcase_0204 | No. of topic : 1 | kill -15 | Async | 1 | 0 |
testcase_0205 | No. of topic : 1 | kill -15 | Sync | -1 | 1 |
testcase_0206 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
testcase_0207 | No. of topic : 1 | kill -15 | Sync | 1 | 1 |
testcase_0208 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
testcase_0209 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
Replication with Follower Failure |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0251 | No. of topic : 1 | kill -15 | Sync | -1 | 0 |
testcase_0252 | No. of topic : 1 | kill -15 | Async | -1 | 0 |
testcase_0253 | No. of topic : 1 | kill -15 | Sync | 1 | 0 |
testcase_0254 | No. of topic : 1 | kill -15 | Async | 1 | 0 |
testcase_0255 | No. of topic : 1 | kill -15 | Sync | -1 | 1 |
testcase_0256 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
testcase_0257 | No. of topic : 1 | kill -15 | Sync | 1 | 1 |
testcase_0258 | No. of topic : 1 | kill -15 | Async | 1 | 1 |
testcase_0259 | No. of topic : 1 | kill -15 | Async | -1 | 1 |
Replication with Simulated Leader GC Pause |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_0301 | No. of topic : 1 | kill -SIGSTOP | Sync | -1 | 0 |
testcase_0302 | No. of topic : 1 | kill -SIGSTOP | Async | -1 | 0 |
testcase_0303 | No. of topic : 1 | kill -SIGSTOP | Sync | 1 | 0 |
testcase_0304 | No. of topic : 1 | kill -SIGSTOP | Async | 1 | 0 |
testcase_0305 | No. of topic : 1 | kill -SIGSTOP | Sync | -1 | 1 |
testcase_0306 | No. of topic : 1 | kill -SIGSTOP | Async | -1 | 1 |
testcase_0307 | No. of topic : 1 | kill -SIGSTOP | Sync | 1 | 1 |
testcase_0308 | No. of topic : 1 | kill -SIGSTOP | Async | 1 | 1 |
testcase_0309 | No. of topic : 1 | kill -SIGSTOP | Async | -1 | 1 |
Log Retention Testing (Size) |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_4001 | No. of topic : 2 | kill -15 | Sync | -1 | 0 |
testcase_4002 | No. of topic : 2 | kill -15 | Sync | 1 | 0 |
testcase_4003 | No. of topic : 2 | kill -15 | Sync | -1 | 1 |
testcase_4004 | No. of topic : 2 | kill -15 | Sync | 1 | 1 |
testcase_4005 | No. of topic : 2 | kill -15 | Async | -1 | 0 |
testcase_4006 | No. of topic : 2 | kill -15 | Async | 1 | 0 |
testcase_4007 | No. of topic : 2 | kill -15 | Async | -1 | 1 |
testcase_4008 | No. of topic : 2 | kill -15 | Async | 1 | 1 |
Log Retention Testing (Size) |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_4011 | No. of topic : 2 | kill -15 | Sync | -1 | 0 |
testcase_4012 | No. of topic : 2 | kill -15 | Sync | 1 | 0 |
testcase_4013 | No. of topic : 2 | kill -15 | Sync | -1 | 1 |
testcase_4014 | No. of topic : 2 | kill -15 | Sync | 1 | 1 |
testcase_4015 | No. of topic : 2 | kill -15 | Async | -1 | 0 |
testcase_4016 | No. of topic : 2 | kill -15 | Async | 1 | 0 |
testcase_4017 | No. of topic : 2 | kill -15 | Async | -1 | 1 |
testcase_4018 | No. of topic : 2 | kill -15 | Async | 1 | 1 |
Mirror Maker |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_5001 | No. of topic : 1 | None | Sync | -1 | 0 |
testcase_5002 | No. of topic : 1 | kill -SIGTERM | Sync | -1 | 0 |
testcase_5003 | No. of topic : 1 | kill -SIGTERM | Async | -1 | 1 |
testcase_5004 | No. of topic : 1 | kill -SIGTERM | Async | 1 | 1 |
testcase_5005 | No. of topic : 2 | kill -SIGTERM | Async | -1 | 1 |
testcase_5006 | No. of topic : 2 | kill -SIGTERM | Async | 1 | 1 |
Migration Tool |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_9001 | No. of topic : 1 | None | Sync | -1 | 0 |
testcase_9003 | No. of topic : 1 | kill -SIGTERM | Async | -1 | 1 |
testcase_9004 | No. of topic : 1 | kill -SIGTERM | Async | 1 | 1 |
testcase_9005 | No. of topic : 2 | kill -SIGTERM | Async | -1 | 1 |
testcase_9006 | No. of topic : 2 | kill -SIGTERM | Async | 1 | 1 |
Others |
|
|
|
|
|
Test Case ID | Test Parameters | Failure | Sync/Async | acks | comp |
testcase_9051 | No. of topics: 300 | kill -SIGTERM | Sync | -1 | 0 |
testcase_9061 | No. of topics: 20 | kill -SIGTERM | Async | -1 | 1 |
testcase_9081 | No. of topic : 2 | None | Async | -1 | 1 |
FAQ
How Do I Run A Quick Sanity Test?
The Sanity test shell script will execute the following 2 test cases: * testcase_0001 (basic test without Broker failure) * testcase_1 (basic test with Leader failure) In <kafka_home>/system_test $ ./run_sanity.sh
How Do I Run A Full Regression Test?
To execute System Test:
<kafka_home>/system_test/ $ python –u –B system_test_runner.py 2>&1 | tee system_test_output_`date +%s`.log -u : python interpreter argument for unbuffering -B : python interpreter argument not to generate binary file
- Notes
1. A template for a full regression test with all testcases is provided as system_test/testcase_to_run_all.json. Please copy this file into testcase_to_run.json to execute all test cases.
2. Otherwise, you may edit system_test/testcase_to_run.json to add / remove test cases to run
3. Edit system_test/logging.conf to turn ON/OFF System Test debug messages (NOT Kafka code log4j messages)
4. If you are running the System Test in your local box, there is no need to make any changes to the cluster_config.json
5. You may change any properties' values inside <xxxx_testsuite>/testcase_xxxx/testcase_xxxx_properties.json to see different behavior. (eg. Changing "bounce_broker": "false" => "true" OR "replica_factor": "3" => "2", etc)