Page History

Apache GitHub Pull Request

https://github.com/apache/knox/pull/365

Performance test tool design

The general goal is to have an extendable performance test framework that will drive the Knox Gateway configured by different use cases.

...

• src/main/java
  
  • o.a.k.g.p.t.PerformanceTestRunner - this is the entry point of the tool. This class comes with a main method that reads the given configuration file and executes all enabled use-case runners
  • o.a.k.g.p.t.ResponseTimeCache - this class acts as a holder of response times and shared between the worker threads (which write into the cache) and the report generation threads (reading data from it)
  • o.a.k.g.p.t.reporting.GatewayMetricsReporter - this class generates the human-readable reports in JSON and YAML format in a fixed schedule marked by perf.test.report.generation.periodInSecs 
  • o.a.k.g.p.t.knoxtoken.KnoxTokenUseCaseRunner - this class is responsible for
    • start N worker threads that are acquiring Knox DTs parallel (marked by perf.test.usecase.knoxtoken.numOfThreads)
    • and 2 more threads to
      • renew an already acquired Knox DT
      • do an HDFS ls command using an already acquired Knox DT

  • o.a.k.g.p.t.knoxtoken.KnoxTokenWorkerThread - this represents the job that actually acquires/renews/uses Knox DTs. The renew/use actions are running only on 1-1 thread and they wait 2 times more time between two subsequential calls then simply executing the acquire action. In other words, by default, a worker thread acquires a Knox DT between every 5 to 10 seconds (on N threads) whereas a worker thread which renews/uses a previously acquired Knox token waits between 10-20 seconds.

  • o.a.k.g.p.t.knoxtoken.KnoxTokenCache - stores the already acquired Knox DTs (if the number of DTs reaches 500 the cache is cleaned automatically)
• src/test/resources
  • performance.test.configuration.properties - contains the above-described configuration file
  • performanceTest-log4j.properties - the Log4j configuration of the tool. By default, it prints log messages on the STDOUT as well as writes them into target/logs/performanceTest.log

Knox gateway requisites

The performance test tool tries to connect to a Knox instance to

...

If you plan to have this topology with a different name (or you already have one that uses JWT and has WEBHDFS), please update the perf.test.usecase.knoxtoken.topology.tokenbased configuration accordingly.

How to run

Running the performance tool is as simple as running the following Maven command in the project root:

...

perf.test.usecase.knoxtoken.numOfThreads=10
perf.test.usecase.knoxtoken.testDurationInSecs=21600

Test results

The JSON/YAML test results are generated under target/testResults/[json|yaml]:

...