Page History

...

#!/usr/bin/python

"""
Description : Split JTL file into a comma delimited CVS
by          : Oliver Erlewein (c)2008
Date        : 04.02.2008
Lang        : Python 2.4+

JMeter JTL field contents:

Attribute & Content
by      Bytes
de      Data encoding
dt      Data type
ec      Error count (0 or 1, unless multiple samples are aggregated)
hn      Hostname where the sample was generated
lb      Label
lt      Latency = time to initial response (milliseconds) - not all samplers support this
na      Number of active threads for all thread groups
ng      Number of active threads in this group
rc      Response Code (e.g. 200)
rm      Response Message (e.g. OK)
s       Success flag (true/false)
sc      Sample count (1, unless multiple samples are aggregated)
t       Elapsed time (milliseconds)
tn      Thread Name
ts      timeStamp (milliseconds since midnight Jan 1, 1970 UTC)
"""

import sys
import string
import re
import datetime
import time

startTime = time.time()
cnt                     = 0
cnt2                    = 0
failCnt                 = 0
reCompile = re.compile("\s([^\s]*?)=\"(.*?)\"")
delimiterCharacterOut = ","

def writeCSVLine(line):
    x = reCompile.findall(line)
    a = dict((row[0], row[1]) for row in x)
    try:
        a['ts'] = str(int(int(a['ts'])/1000))
        x = str(datetime.datetime.fromtimestamp(float(a['ts'])))[0:19]
        b = a['ts'] + ",\"" + x + "\"," + a['t'] + "," + a['lt'] + ",\"" + a['s'] + "\",\"" + a['lb'] + "\"," + a['rc'] + ",\"" + a['rm'] + "\",\"" + a['tn'] + "\",\"" + a['dt'] + "\"," + a['by'] + "\n"
    except:
        return -1
    o.write(b)
    return 1

print "Splitting JTL file"

try:
    runArgv             = sys.argv                      # Save the command line
    jtlInfile           = str(sys.argv[1])              # Name of JTL input file
    cvsOutfile          = str(sys.argv[2])              # Name of CVS output file
    reFilter            = str(sys.argv[3])              # Filter the labels (lb) for the filter
except:
    print "Error: Input format: <input file> <output file> <Filter by regular expression>"
    raise

try:
    f = open(jtlInfile, "r")
    o = open(cvsOutfile, "w")
except:
    raise

print "Filtering on regular expression : " + reFilter 
cmpFilter = re.compile(reFilter)

for line in f:
    try:
        if cmpFilter.search(line):
           returnVal = writeCSVLine(line)
           if returnVal < 0:
              failCnt += 1
           else:
              cnt2 += 1
    except:
        print 'Error in line : ', cnt, line
        raise

    cnt += 1

endTime = time.time()
print "Time taken                    : ", str(endTime-startTime)
print "Lines processed              : ", cnt
print "Lines that passed the filter : ", cnt2
print "Lines skipped (error?)       : ", failCnt

f.close()
o.close()

Generating charts with Perl

(Christoph Meissner) When it comes to exploit jmeters logfiles generated by many clients over a long testing period one might run into time consuming efforts putting all data together into charts. To reduce effort I wrote a Perl script that parses any number of jmeter logs and aggregates the data into several different charts (examples follow below):

Here are example charts showing a test against an address validation tool stressed with 25 active threads:

Stacked chart showing cusps of response times

Measure points are shown at the x-axis.

attachment:cm_ChartStacked.png

Stacked chart showing response times in %

attachment:cm_ChartStackedPct.png

Chart opposing total response times to total threads

x-axis shows the timestamps of the test, left y-axis holds response times, right y-axis holds number of active threads (it is also possible to display a 1 standard deviation line for the response times (see script below)).

attachment:cm_entire_users.png

Chart opposing total response times to total throughput

x-axis shows the timestamps of the test, left y-axis holds response times, right y-axis holds throughput (it is also possible to display a 1 standard deviation line for the response times (see script below))

attachment:cm_entire_throughput.png

Chart opposing response times of a single measure point to active threads

attachment:cm_AdressService_users.png

Chart opposing response times of a single measure point to throughput

attachment:cm_AdressService_throughput.png

Configure jmeters testplan

To make jmeter to generate appropriate log files you will have to configure your testplan properly.

One task will be to name your requests (the pipettes) in a way that allows the script to cumulate response times, active threads and throughput together into measure points. Also you will have to tell jmeter to log particular data.

For both requirements look at the example screenshot, please:

attachment:cm_jmeter.png

1. insert an 'Aggregate Report' Listener to your testplan (top level). 2. enter a filename where to save the results 3. invoke 'Configure' and check the boxes as shown. 4. Change the names of your requests.

(for instance in this test all requests for images were named as 'GIF', all requests for java scripts were named as 'JS'. Special points of interest also get their appropriate name (eg. 'Login', 'Address Search', ...))

This step is necessary because the script collects data into labels of the same name. This way you make sure that you get a chart that will show the response times for all images or all 'Login' or all 'Address Searches'.

The script

The attachment:jmetergraph.pl requires Chart 2.4.1 to be installed (http://search.cpan.org/~chartgrp/Chart-2.4.1/).

You can call the script in this way:

perl jmetergraph.pl [-stddev] [-range] <jtl file 1> [ ... <jtl file n>]

If you pass -stddev the graphs will show the normal standard deviation of the response times, too.

attachment:cm_entire_users_stddev.png

If you pass -range then a bar chart will be generated showing the average range of the reponse times.

attachment:cm_entire_users_range.png