This page describes how to perform stress test against OpenWhisk with nGrinder.

If you have no idea what nGrinder is, please refer to this page: [Getting Started with nGrinder]


Throughput test with 1 action.

You can create a script with OpenWhisk apiHost as a URL to test.


You can use the following script to test OpenWhisk.

invoke-action.groovy
import static net.grinder.script.Grinder.grinder
import static org.junit.Assert.*
import static org.hamcrest.Matchers.*
import net.grinder.plugin.http.HTTPRequest
import net.grinder.plugin.http.HTTPPluginControl
import net.grinder.script.GTest
import net.grinder.script.Grinder
import net.grinder.scriptengine.groovy.junit.GrinderRunner
import net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess
import net.grinder.scriptengine.groovy.junit.annotation.BeforeThread
import org.junit.Before
import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith
 
import java.util.Date
import java.util.List
import java.util.ArrayList
import java.util.Random
 
import HTTPClient.Cookie
import HTTPClient.CookieModule
import HTTPClient.HTTPResponse
import HTTPClient.NVPair
 
@RunWith(GrinderRunner)
class TestRunner {
 
    public static GTest test
    public static HTTPRequest request
    public static NVPair[] headers = []
    public static NVPair[] params = []
 
 
    @BeforeProcess
    public static void beforeProcess() {
        HTTPPluginControl.getConnectionDefaults().timeout = 60000
        test = new GTest(1, "openwhisk.incubator.apache.org")
        request = new HTTPRequest()
 
        // Set headers
        List<NVPair> headerList = new ArrayList<NVPair>()
        headerList.add(new NVPair("Content-Type", "application/json"))
  
        // This is a Authorization header for the namespace, whisk.system.
        headerList.add(new NVPair("Authorization", "Basic Nzg5YzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOmFiY3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A="))
 
        headers = headerList.toArray()
        grinder.logger.info("before process.");
    }
 
    @BeforeThread
    public void beforeThread() {
        test.record(this, "test")
        grinder.statistics.delayReports=true;
        grinder.logger.info("before thread.");
    }
 
    @Before
    public void before() {
        request.setHeaders(headers)        
        grinder.logger.info("before thread. init headers and cookies");
    }
 
    @Test
    public void test(){
        HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/whisk.system/actions/noopThroughput?blocking=true&result=true", params)        
 
        assertThat(result.statusCode, is(200));
 
    }
}


There's no big difference with default codes.

It just adds Authorization header and invokes noopThroughput action as a blocking call.

Regarding noopThroughput action, I just created it in advance just like this: https://github.com/apache/incubator-openwhisk/blob/master/tests/performance/preparation/create.sh#L29


You can validate your script before saving.

If there's no issue, you can see 200 OK result in the logs

Once the script is created, you are ready to run a test.

Move to Performance Test and create a test with the script you've just created.



Throughput test with 100 actions.


Once you test with 1 action, you may want to test with more number of actions.

Since you can define the test with the codes, it's very easy to do.


You need to create 100 actions(noopThroughput0 ~ noopThroughput99) in advance and randomly pick them in the test method.

test script
@Test
public void test(){
  Random rand = new Random();
  int randActionNum = rand.nextInt(100);
 
  HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/whisk.system/actions/noopThroughput" + randActionNum +"?blocking=true&result=true", params)        
 
  assertThat(result.statusCode, is(200));
 
}



Throughput test with multiple namespaces and actions.

Surely you can also test with multiple namespaces and actions.

You just need to do few more things in advance.


First, you need to create multiple namespaces(subjects).

Please refer to the following bash script. It should be executed in OPENWHISK_HOME directory.

Create 100 namespaces
# UUID and Auth_Key can be anything.
UUID=11112ded-78ab-4cb8-afc7-452cc7d62
AUTH_KEY=AXg2egwVdeAaXKHTErTdBJKbY2jnLJzXWuyrMzw7vDFtbjUOkdhgDVqzWmqlfUIz
 
# You can control the number of users to be created.
for i in {0..100}
do
echo "Creating user $i"
 
NAMESPACE=stub_user_$i
AUTH=$UUID$(printf "%03d" $i):$AUTH_KEY
echo "$NAMESPACE: $AUTH"


# Delete the namespace if it already exists.
bin/wskadmin user delete $NAMESPACE
bin/wskadmin user create -u $AUTH $NAMESPACE
 
# Create a test action in the namespace.
# You can create more actions here.
wsk action update /$NAMESPACE/noopThroughput0 noop.js -i --apihost https://10.10.10.10 -u $AUTH
wsk action invoke /$NAMESPACE/noopThroughput0 --result -i --apihost https://10.10.10.10 -u $AUTH


wsk action update /$NAMESPACE/noopThroughput1 noop.js -i --apihost https://10.10.10.10 -u $AUTH
wsk action invoke /$NAMESPACE/noopThroughput1 --result -i --apihost https://10.10.10.10 -u $AUTH
done


Once namespace and action creation are done, you can use the following script to test with them.

Test script
@Test
public void test(){        
  Random rand = new Random();
 
  int randUserNum = rand.nextInt(99); // the user number
  int randActionNum = rand.nextInt(2); // the action number
 
  // This UUID and KEY should be same with the one in the bash script.
  String uuid = '11112ded-78ab-4cb8-afc7-452cc7d62' + String.format("%03d", randUserNum)
  String key = 'AXg2egwVdeAaXKHTErTdBJKbY2jnLJzXWuyrMzw7vDFtbjUOkdhgDVqzWmqlfUIz'
 
  List<NVPair> headerList = new ArrayList<NVPair>()
  headerList.add(new NVPair("Content-Type", "application/json"))
  headerList.add(HTTPPluginControl.getHTTPUtilities().basicAuthorizationHeader(uuid, key))
  NVPair[] headers = headerList.toArray()
 
  // Invoke randomly chosen action in randomly chosen namespace.
  HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/stub_user_" + randUserNum + "/actions/noopThroughput"+randActionNum+"?blocking=true&result=true", params, headers)        
 
  assertThat(result.statusCode, is(200));
}

Run a test with multiple runtimes or different action memory setting.

Similarly you can create actions with multiple runtimes or with different memory setting.


  • No labels