...
5. You can also use the following package that mixes Excel Macro, java transformation and xsl transformation of the .jtl files. It automatically generates Excel graphs using a Macro. The package is available here :
attachment:scripts_jmeter.zip
Testing a High Volume Webservice: Simple Data Writer, Bash scripts & Excel Graphs
Wiki Markup |
---|
For a telecoms project, JMeter was used to test a new high-availability webservice/integration layer, exposed to a few internal clients (see also \[:../UserManual/BuildWSTest:BuildWSTest\]). Several instances of JMeter represented the different clients and varying load profiles. The main problems were: *The default JMeter XML output is too verbose, and the files grew too enormous, so the *Simple Data Writer* was used to produce much friendlier CSV results files. I wanted to show throughput & response times in blocks of 1 minute; *JMeter's inbuilt Graph function was not sufficient to process the volume of data. An example of the type of graph required follows. The reader will see that the JMeter test plan produced variations in load (normal, high & spike), and the response times were quite well-behaved. If the integration layer was performing poorly, the graph would show inconsistent throughput and fluctuating response times. Unix/Linux [BR] attachment:throughput-graph.png [BR] |
Excel Throughput Graph
Script: attachment:jtlmin.sh.txt BR JMeter's output graph is too granular to depict throughput for extended test intervals (anything from 2 to 24 hours). An Excel constraint its maximum of 65536 rows. So to produce a throughput graph, JTL files of ~100k rows should be summarized into increments of 1 minute (or 2,5,n minutes depending on requirements).
BRFor each minute: throughput = count of transactions in that minute ; response time = average of 'elapsed' values in that minute. BRThe script jtlmin.sh
summarizes large JTL files into 1 minute increments producing an OUT file that can be imported to Excel and a graph produced. The core functionality in jtlmin.sh
is this piece of awk code:
No Format |
---|
# scan a JTL file for records in a specified interval
# and return record count & average response time.
BEGIN {
avgresponse=0; sumresponse=0; trancount=0;
}
{
if(($1 >= lastmin) && ($1 < thismin)) {
trancount++
sumresponse += $2
avgresponse = sumresponse / trancount
}
}
END {
printf("%d %d %d %d",lastmin,sumresponse,trancount,avgresponse);
print " ",strftime("%Y.%b.%d %H:%M",lastmin)
}
|
An example session, using jtlmin.sh
to process a JTL file. The file produced, queryBalance.jtl.OUT
(tab-delimited), can now be used to produce throughput graph. Response times can also be included on the secondary axis, as in the diagram above. These graphs were very good at showing when the integration layer was slow to respond and when throughput varied from the original JMeter plan.
No Format |
---|
$ jtlmin.sh
Usage: jtlmin.sh <filename>
Summarizes JMeter JTL output into 1-minute blocks
$ jtlmin.sh queryBalance.jtl
Processing queryBalance.jtl
$ ls q*
queryBalance.jtl queryBalance.jtl.OUT
$ head queryBalance.jtl.OUT
/c/jmeter/performance/Myserver/output/queryBalance.jtl
unixtime date time thruput(tpm) response(ms)
1160354940 2006.Oct.09 13:49 65 0
1160355000 2006.Oct.09 13:50 0 0
1160355060 2006.Oct.09 13:51 0 0
1160355120 2006.Oct.09 13:52 56 0
1160355180 2006.Oct.09 13:53 98 108
1160355240 2006.Oct.09 13:54 84 125
1160355300 2006.Oct.09 13:55 0 0
1160355360 2006.Oct.09 13:56 0 0
|
Simple Data Writer JTL files
Wiki Markup |
---|
JMeter's \[http://jakarta.apache.org/jmeter/usermanual/build-monitor-test-plan.html Simple Data Writer\] produces JTL output files which convey the same information as the default XML output, but the CSV format is much denser. Example: |
No Format |
---|
queryBalance.jtl
timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,bytes,grpThreads,allThreads,URL
1158477785863,351,SL_queryBalance,200,,queryBalance 1-1,text,true,87,7,7,http://123.45.67.89:8080/WebService
1158477785953,291,SL_queryBalance,200,,queryBalance 1-4,text,true,87,7,7,http://123.45.67.89:8080/WebService
1158477785883,431,SL_queryBalance,200,,queryBalance 1-2,text,true,87,8,8,http://123.45.67.89:8080/WebService
1158477786013,301,SL_queryBalance,200,,queryBalance 1-5,text,true,87,8,8,http://123.45.67.89:8080/WebService
1158477785883,481,SL_queryBalance,200,,queryBalance 1-3,text,true,87,9,9,http://123.45.67.89:8080/WebService
1158477786113,331,SL_queryBalance,200,,queryBalance 1-6,text,true,87,10,10,http://123.45.67.89:8080/WebService
1158477786183,301,SL_queryBalance,200,,queryBalance 1-7,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786404,120,SL_queryBalance,200,,queryBalance 1-10,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786334,200,SL_queryBalance,200,,queryBalance 1-9,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786254,290,SL_queryBalance,200,,queryBalance 1-8,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786474,140,SL_queryBalance,200,,queryBalance 1-11,text,true,87,12,12,http://123.45.67.89:8080/WebService
...
|
Conversion of JMeter timestamps
Script: attachment:utime2ymd.txt BR The first field of a JTL output file is a Unix timestamp extended to milliseconds. The above script jtltotals.sh
calls another script utime2ymd
to convert start & end times into year-month-day.hour-min-sec (yyyymmdd.HHMMss). Usually the JTL timestamps are adjusted for your local timezone (eg. GMT plus or minus a few hours). The utime2ymd
script uses the local timezone by default, but can also provide GMT values – useful for converting x-thousand elapsed seconds into hhmmss. Example of usage:
No Format |
---|
$ utime2ymd
Usage: utime2ymd <timestamp> [local|gmt]
Convert 10-digit Unix timestamp to yyyymmdd.hhmmss format
use local time zone (default) or UTC/GMT
$ utime2ymd 1158477785863
20060917.192305 local
$ utime2ymd 3601 gmt
19700101.010001 gmt
|
Overview of Several Output files
Wiki Markup |
---|
Script: attachment:jtltotals.sh.txt [BR] After a test run, all the JTL output files were gathered together (20 or so files) in a bunch of subdirectories. The analysis was conducted on a Windows PC with MinGW/MinSYS and a few other tools (msys-dtk, gnu bc, gnu paste, gVim). For an overview of total vs. projected throughput, I used the shell script {{jtltotals.sh}} (a bit kludgy but hey I'm a tester not a developer!). It collates \[total throughput, start time, end time, time elapsed, average response time\] for each output file. This script will produce a (comma-delimited) file 'jtl-file-totals.txt'. A sample of output is shown below. |
BASH shell scripts were used extensively for analysis, and preparing data for Excel graphs. [BR] attachment:throughput-graph.png [BR] |
Simple Data Writer JTL files
Wiki Markup |
---|
JMeter's \[http://jakarta.apache.org/jmeter/usermanual/build-monitor-test-plan.html Simple Data Writer\] produces JTL output files which convey the same information as the default XML output, but the CSV format is much denser. Example: |
No Format |
---|
queryBalance.jtl
timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,bytes,grpThreads,allThreads,URL
1158477785863,351,SL_queryBalance,200,,queryBalance 1-1,text,true,87,7,7,http://123.45.67.89:8080/WebService
1158477785953,291,SL_queryBalance,200,,queryBalance 1-4,text,true,87,7,7,http://123.45.67.89:8080/WebService
1158477785883,431,SL_queryBalance,200,,queryBalance 1-2,text,true,87,8,8,http://123.45.67.89:8080/WebService
1158477786013,301,SL_queryBalance,200,,queryBalance 1-5,text,true,87,8,8,http://123.45.67.89:8080/WebService
1158477785883,481,SL_queryBalance,200,,queryBalance 1-3,text,true,87,9,9,http://123.45.67.89:8080/WebService
1158477786113,331,SL_queryBalance,200,,queryBalance 1-6,text,true,87,10,10,http://123.45.67.89:8080/WebService
1158477786183,301,SL_queryBalance,200,,queryBalance 1-7,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786404,120,SL_queryBalance,200,,queryBalance 1-10,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786334,200,SL_queryBalance,200,,queryBalance 1-9,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786254,290,SL_queryBalance,200,,queryBalance 1-8,text,true,87,11,11,http://123.45.67.89:8080/WebService
1158477786474,140,SL_queryBalance,200,,queryBalance 1-11,text,true,87,12,12,http://123.45.67.89:8080/WebService
...
|
Excel Throughput Graph
Script: attachment:jtlmin.sh.txt BR JMeter's output graph is too granular to depict throughput for extended test intervals (anything from 2 to 24 hours). An Excel constraint its maximum of 65536 rows. So JTL files of ~100k rows should be summarized into increments of 1 minute (or 2,5,n minutes depending on requirements).
BRFor each minute: throughput = count of transactions in that minute ; response time = average of 'elapsed' values in that minute. BRThe script jtlmin.sh
summarizes large JTL files into 1 minute increments producing an OUT file that can be imported to Excel and a graph produced. The core functionality in jtlmin.sh
is this piece of awk code:
No Format |
---|
# scan a JTL file for records in a specified interval
# and return record count & average response time.
BEGIN {
avgresponse=0; sumresponse=0; trancount=0;
}
{
if(($1 >= lastmin) && ($1 < thismin)) {
trancount++
sumresponse += $2
avgresponse = sumresponse / trancount
}
}
END {
printf("%d %d %d %d",lastmin,sumresponse,trancount,avgresponse);
print " ",strftime("%Y.%b.%d %H:%M",lastmin)
}
|
An example session, using jtlmin.sh
to process a JTL file. The file produced, queryBalance.jtl.OUT
(tab-delimited), can now be used to produce throughput graph. Response times can also be included on the secondary axis, as in the diagram above. These graphs were very good at showing when the integration layer was slow to respond and when throughput varied from the original JMeter plan.
No Format |
---|
$ jtlmin.sh
Usage: jtlmin.sh <filename>
Summarizes JMeter JTL output into 1-minute blocks
$ jtlmin.sh queryBalance.jtl
Processing queryBalance.jtl
$ ls q*
queryBalance.jtl queryBalance.jtl.OUT
$ head queryBalance.jtl.OUT
/c/jmeter/performance/Myserver/output/queryBalance.jtl
unixtime date time thruput(tpm) response(ms)
1160354940 2006.Oct.09 13:49 65 0
1160355000 2006.Oct.09 13:50 0 0
1160355060 2006.Oct.09 13:51 0 0
1160355120 2006.Oct.09 13:52 56 0
1160355180 2006.Oct.09 13:53 98 108
1160355240 2006.Oct.09 13:54 84 125
1160355300 2006.Oct.09 13:55 0 0
1160355360 2006.Oct.09 13:56 0 0
|
Overview of Several Output files
Wiki Markup |
---|
Script: attachment:jtltotals.sh.txt [BR] After a test run, all the JTL output files were gathered together (20 or so files) in a bunch of subdirectories. The analysis was conducted on a Windows PC with MinGW/MinSYS and a few other tools (msys-dtk, gnu bc, gnu paste, gVim). For an overview of total vs. projected throughput, I used the shell script {{jtltotals.sh}} (a bit kludgy but hey I'm a tester not a developer!). It collates \[total throughput, start time, end time, time elapsed, average response time\] for each output file. This script will produce a (comma-delimited) file 'jtl-file-totals.txt'. A sample of output is shown below. |
No Format |
---|
jtl-file-totals.txt
JMeter-Output-file,total-throughput,start,end,elapsed-sec,elapsed-hms,response-av
WebGUI/output.1/queryFCNs.jtl,33,20061103.105342 local,20061103.105830 local,288,00:04:48,225.59
WebGUI/output.1/queryPackages.jtl,55,20061103.105342 local,20061103.105555 local,133,00:02:13,234.06
WebGUI/output.2/queryFCNs.jtl,42,20061103.113435 local,20061103.114155 local,440,00:07:20,212.12
WebGUI/output.2/queryPackages.jtl,59,20061103.113435 local,20061103.113737 local,182,00:03:02,238.78
WebGUI/output.3/queryPackages.jtl,272,20061103.121135 local,20061103.122042 local,547,00:09:07,260.03
Myserver/output/applyDebit.jtl,22219,20060912.154822 local,20060912.162945 local,2483,00:41:23,1265.12
Myserver/output/queryBalance.jtl,360,20061009.134916 local,20061009.150914 local,4798,01:19:58,96.31
total,23040,,,,,
|
Conversion of JMeter timestamps
Script: attachment:utime2ymd.txt BR The first field of a JTL output file is a Unix timestamp extended to milliseconds. The above script jtltotals.sh
calls another script utime2ymd
to convert start & end times into year-month-day.hour-min-sec (yyyymmdd.HHMMss). Usually the JTL timestamps are adjusted for your local timezone (eg. GMT plus or minus a few hours). The utime2ymd
script uses the local timezone by default, but can also provide GMT values – useful for converting x-thousand elapsed seconds into hhmmss. Example of usage:
No Format |
---|
$ utime2ymd
Usage: utime2ymd <timestamp> [local|gmt]
Convert 10-digit Unix timestamp to yyyymmdd.hhmmss format
use local time zone (default) or UTC/GMT
$ utime2ymd 1158477785863
20060917.192305 local
$ utime2ymd 3601 gmt
19700101.010001 gmt |
No Format |
jtl-file-totals.txt
JMeter-Output-file,total-throughput,start,end,elapsed-sec,elapsed-hms,response-av
WebGUI/output.1/queryFCNs.jtl,33,20061103.105342 local,20061103.105830 local,288,00:04:48,225.59
WebGUI/output.1/queryPackages.jtl,55,20061103.105342 local,20061103.105555 local,133,00:02:13,234.06
WebGUI/output.2/queryFCNs.jtl,42,20061103.113435 local,20061103.114155 local,440,00:07:20,212.12
WebGUI/output.2/queryPackages.jtl,59,20061103.113435 local,20061103.113737 local,182,00:03:02,238.78
WebGUI/output.3/queryPackages.jtl,272,20061103.121135 local,20061103.122042 local,547,00:09:07,260.03
Myserver/output/applyDebit.jtl,22219,20060912.154822 local,20060912.162945 local,2483,00:41:23,1265.12
Myserver/output/queryBalance.jtl,360,20061009.134916 local,20061009.150914 local,4798,01:19:58,96.31
total,23040,,,,,
|
Extract from JMeter Test Plan (JMX file)
Script: attachment:jmxparse.sh.txt BR Another possibly useful tool which will give a text based summary of what's in your JMeter JMX script. Mainly uses grep and sed.
...
No Format |
---|
//@testname| //elementProp/@name| //elementProp/stringProp[@name="Argument.value"]/text()| //ThreadGroup/@enabled| //stringProp[@name="RunTime.seconds"]/text()| //stringProp[@name="throughput"]/text()| //stringProp[@name="filename"]/text() |
About
...
the Bash scripts
Wiki Markup |
---|
I used \[http://www.mingw.org/download.shtml MinGW\] is a subset of Cygwin, but I prefer it because it's a lightweight install on Windows, and it has a nicer interface (rxvt), but it provides] with the familiar \*nix text processing tools. Recommended packages/versions are: |
...
; here's a list *\[http://prdownloads.sourceforge.net/mingw/MinGW-3.1.0-1.exe?download MinGW |
...
\] |
...
, \[http://prdownloads.sourceforge.net/mingw/MSYS-1.0.10.exe?download MSYS |
...
\] |
...
, \[http://prdownloads.sourceforge.net/mingw/msysDTK-1.0.1.exe?download |
...
MSys-DTK\], \[http://sourceforge.net/project/showfiles.php?group_id=23617&package_id=26968 |
...
Gnu-bc\], \[http://prdownloads.sourceforge.net/mingw/join_paste_textutils-2.1-MSYS.tar.bz2?download |
...
paste |
...
\] |
...
, \[http://www.bzip.org/1.0.3/bzip2-103 |
...
-x86-linux26 bzip2\], \[ftp://ftp.vim.org/pub/vim/pc/gvim70.exe gVim 7.0\] |
...
Wiki Markup |
---|
*Windows alternatives (with large footprint) are \[http://www.cygwin.com Cygwin\] or, \[http://www.microsoft.com/technet/interopmigration/unix/sfu/default.mspx windowsWindows SFU\], or even ubuntu linux on a Virtual PC *Or use a Linux/Unix OS exclusively to run the above scripts .. |