THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| No Format |
|---|
-Dfile_format.testlog=2.0 |
Wiki Markup
The above settings can also be defined as global properties in the JMeter.properties configuration file.
...
The graphs generated can be quite useful in visualizing performance of websites. See the graph below for an example of one such dummy website. The JMeter test that generated the data imposed a steady "normal load" component, but also imposed two short-term, high volume "surge loads" spread 10 minutes apart. (This was done using a separate JMeter thread group).
attachment:DummyWebsitePerformance.JPG!
XSL script to extract Jmeter logs
...
5. You can also use the following package that mixes Excel Macro, java transformation and xsl transformation of the .jtl files. It automatically generates Excel graphs using a Macro. The package is available here :
attachment: scripts_jmeter.zip
Summarizing Huge Datasets
Shell Script to Aggregate Per Minute
As a software tester, sometimes you are called upon to performance test a web service (see BuildWSTest) and present results in a nice chart to impress your manager. JMeter is commonly used to thrash the server and produce insane amounts of throughput data. If you're running 1000 tpm this can be rather a lot of data (180,000 transactions for a 3 hour test run). Even using the Simple Data Writer, this is beyond the capability of JMeter's inbuilt graphics package and is too much to import to Wiki Markup
My solution is to group throughput per minute and average transaction time for each minute. Attached below is a Bash script for processing a JTL log file from JMeter. It reduces a 3-hour test run to 180 data points which is much easier to represent with a chart program such as Excel. BR
The script uses a few neat awk tricks, such as: BR
- Rounding unix timestamps to nearest minute
- Collect timestamps grouped by minute
- Convert unix timestamp to YYYY-MM-dd etc.
- Print Throughput for a minute increment
- Print Average response time for a minute increment
- Do all of the above in an efficient single pass through awk (this was the hardest bit!)
Script: attachment: jtlmin.sh.txt
BR An example session, using jtlmin.sh to process a JTL file. The file produced, queryBalance.jtl.OUT (tab-delimited), can now be used to produce throughput graph. Response times can also be included on the secondary axis, as in the diagram above. These graphs were very good at showing when the integration layer was slow to respond and when throughput varied from the original JMeter plan.
| No Format |
|---|
$ jtlmin.sh Usage: jtlmin.sh <filename> Summarizes JMeter JTL output into 1-minute blocks $ jtlmin.sh queryBalance.jtl Processing queryBalance.jtl $ ls q* queryBalance.jtl queryBalance.jtl.OUT $ head queryBalance.jtl.OUT /c/jmeter/performance/Myserver/output/queryBalance.jtl unixtime date time thruput(tpm) response(ms) 1160354940 2006.Oct.09 13:49 65 0 1160355000 2006.Oct.09 13:50 0 0 1160355060 2006.Oct.09 13:51 0 0 1160355120 2006.Oct.09 13:52 56 0 1160355180 2006.Oct.09 13:53 98 108 1160355240 2006.Oct.09 13:54 84 125 1160355300 2006.Oct.09 13:55 0 0 1160355360 2006.Oct.09 13:56 0 0 |
Script: attachment: jtlmin.sh.txt BR
NB, here's a script to convert JMeter's Unix timestamps:
Script: attachment:utime2ymd.txt BR
Java Class to Quickly Summarize JMeter Results
Wiki Markup
Java Class: attachment: JMeterSummary.java
Sample Ouput:
| No Format |
|---|
All Urls:
cnt: 333, avg t: 535 ms, max t: 30755 ms, min t: 10 ms, result codes: {200=291, 302=42}, failures: 0, cnt by time: [0.0 s - 0.5 s = 312, 0.5 s - 1.0 s = 16, 30.5 s - 31.0 s = 5]
avg conn: 17 ms, max conn: 120 ms, min conn: 0 ms, elapsed seconds: 71 s, cnt per second: 5
URL: /
cnt: 27, avg t: 44 ms, max t: 90 ms, min t: 20 ms, result codes: {200=27}, failures: 0, cnt by time: [0.0 s - 0.5 s = 27]
URL: /home.html
cnt: 30, avg t: 173 ms, max t: 581 ms, min t: 70 ms, result codes: {200=30}, failures: 0, cnt by time: [0.0 s - 0.5 s = 27, 0.5 s - 1.0 s = 3]
URL: /news/
cnt: 9, avg t: 93 ms, max t: 120 ms, min t: 70 ms, result codes: {200=9}, failures: 0, cnt by time: [0.0 s - 0.5 s = 9]
URL: /signout.jsp
cnt: 21, avg t: 30 ms, max t: 60 ms, min t: 20 ms, result codes: {302=21}, failures: 0, cnt by time: [0.0 s - 0.5 s = 21]
...
|
...
If you use CSV format logs, this method of summarizing one or more CSV files orders of magnitude faster than importing it into the Aggregate Report Listener. It requires PostgreSQL. Usage:
jtlsummary.sh jtl_csv_result_files...
attachment:jtlsummary.sh
Extracting JTL files to CSV with Python (JMeter 2.3.x)
...
Measure points are shown at the x-axis..
attachment:cm_ChartStacked.png
Stacked chart showing response times in %
attachment:cm_ChartStackedPct.png
Chart opposing total response times to total threads
x-axis shows the timestamps of the test, left y-axis holds response times, right y-axis holds number of active threads (it is also possible to display a 1 standard deviation line for the response times (see script below)).
attachment:cm_entire_users.png
Chart opposing total response times to total throughput
x-axis shows the timestamps of the test, left y-axis holds response times, right y-axis holds throughput (it is also possible to display a 1 standard deviation line for the response times (see script below))
attachment:cm_entire_throughput.png
Chart opposing response times of a single measure point to active threads
attachment:cm_AdressService_users.png
Chart opposing response times of a single measure point to throughput
attachment:cm_AdressService_throughput.png
Configure jmeters testplan
...
For both requirements look at the example screenshot, please:
attachment:cm_jmeter.png
- insert an 'Aggregate Report' Listener to your testplan (top level). 2. enter a filename where to save the results 3. invoke 'Configure' and check the boxes as shown. 4. Change the names of your requests. For instance in my exemplified testplan all requests for images were named as 'GIF', all requests for java scripts were named as 'JS'. Special points of interest also get their appropriate name (eg. 'Login', 'Address Search', ...).
If you don't like particular requests to be accumulated then name them starting with garbage (eg. garbage_gif, garbage_js, ...)
This step is necessary because the script collects data into labels of the same name. This way you make sure that you get a chart that will show the response times for all images or all 'Login' or all 'Address Searches' (except those that start with garbage)
The script
The attachment: jmetergraph.pl requires Chart 2.4.1 to be installed (http://search.cpan.org/~chartgrp/Chart-2.4.1/).
...
If you pass -stddev the graphs will show the normal standard deviation of the response times, too.
attachment:cm_entire_users_stddev.png
If you pass -range then a bar chart will be generated showing the average range of the reponse times.
attachment:cm_entire_users_range.png