This page describes how to perform stress test against OpenWhisk with nGrinder.
If you have no idea what nGrinder is, please refer to this page: [Getting Started with nGrinder]
Throughput test with 1 action.
You can create a script with OpenWhisk apiHost as a URL to test.
You can use the following script to test OpenWhisk.
import static net.grinder.script.Grinder.grinder import static org.junit.Assert.* import static org.hamcrest.Matchers.* import net.grinder.plugin.http.HTTPRequest import net.grinder.plugin.http.HTTPPluginControl import net.grinder.script.GTest import net.grinder.script.Grinder import net.grinder.scriptengine.groovy.junit.GrinderRunner import net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess import net.grinder.scriptengine.groovy.junit.annotation.BeforeThread import org.junit.Before import org.junit.BeforeClass import org.junit.Test import org.junit.runner.RunWith import java.util.Date import java.util.List import java.util.ArrayList import java.util.Random import HTTPClient.Cookie import HTTPClient.CookieModule import HTTPClient.HTTPResponse import HTTPClient.NVPair @RunWith(GrinderRunner) class TestRunner { public static GTest test public static HTTPRequest request public static NVPair[] headers = [] public static NVPair[] params = [] @BeforeProcess public static void beforeProcess() { HTTPPluginControl.getConnectionDefaults().timeout = 60000 test = new GTest(1, "openwhisk.incubator.apache.org") request = new HTTPRequest() // Set headers List<NVPair> headerList = new ArrayList<NVPair>() headerList.add(new NVPair("Content-Type", "application/json")) // This is a Authorization header for the namespace, whisk.system. headerList.add(new NVPair("Authorization", "Basic Nzg5YzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOmFiY3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=")) headers = headerList.toArray() grinder.logger.info("before process."); } @BeforeThread public void beforeThread() { test.record(this, "test") grinder.statistics.delayReports=true; grinder.logger.info("before thread."); } @Before public void before() { request.setHeaders(headers) grinder.logger.info("before thread. init headers and cookies"); } @Test public void test(){ HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/whisk.system/actions/noopThroughput?blocking=true&result=true", params) assertThat(result.statusCode, is(200)); } }
There's no big difference with default codes.
It just adds Authorization header and invokes noopThroughput action as a blocking call.
Regarding noopThroughput action, I just created it in advance just like this: https://github.com/apache/incubator-openwhisk/blob/master/tests/performance/preparation/create.sh#L29
You can validate your script before saving.
If there's no issue, you can see 200 OK result in the logs
Once the script is created, you are ready to run a test.
Move to Performance Test and create a test with the script you've just created.
Throughput test with 100 actions.
Once you test with 1 action, you may want to test with more number of actions.
Since you can define the test with the codes, it's very easy to do.
You need to create 100 actions(noopThroughput0 ~ noopThroughput99) in advance and randomly pick them in the test method.
@Test public void test(){ Random rand = new Random(); int randActionNum = rand.nextInt(100); HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/whisk.system/actions/noopThroughput" + randActionNum +"?blocking=true&result=true", params) assertThat(result.statusCode, is(200)); }
Throughput test with multiple namespaces and actions.
Surely you can also test with multiple namespaces and actions.
You just need to do few more things in advance.
First, you need to create multiple namespaces(subjects).
Please refer to the following bash script. It should be executed in OPENWHISK_HOME directory.
# UUID and Auth_Key can be anything. UUID=11112ded-78ab-4cb8-afc7-452cc7d62 AUTH_KEY=AXg2egwVdeAaXKHTErTdBJKbY2jnLJzXWuyrMzw7vDFtbjUOkdhgDVqzWmqlfUIz # You can control the number of users to be created. for i in {0..100} do echo "Creating user $i" NAMESPACE=stub_user_$i AUTH=$UUID$(printf "%03d" $i):$AUTH_KEY echo "$NAMESPACE: $AUTH" # Delete the namespace if it already exists. bin/wskadmin user delete $NAMESPACE bin/wskadmin user create -u $AUTH $NAMESPACE # Create a test action in the namespace. # You can create more actions here. wsk action update /$NAMESPACE/noopThroughput0 noop.js -i --apihost https://10.10.10.10 -u $AUTH wsk action invoke /$NAMESPACE/noopThroughput0 --result -i --apihost https://10.10.10.10 -u $AUTH wsk action update /$NAMESPACE/noopThroughput1 noop.js -i --apihost https://10.10.10.10 -u $AUTH wsk action invoke /$NAMESPACE/noopThroughput1 --result -i --apihost https://10.10.10.10 -u $AUTH done
Once namespace and action creation are done, you can use the following script to test with them.
@Test public void test(){ Random rand = new Random(); int randUserNum = rand.nextInt(99); // the user number int randActionNum = rand.nextInt(2); // the action number // This UUID and KEY should be same with the one in the bash script. String uuid = '11112ded-78ab-4cb8-afc7-452cc7d62' + String.format("%03d", randUserNum) String key = 'AXg2egwVdeAaXKHTErTdBJKbY2jnLJzXWuyrMzw7vDFtbjUOkdhgDVqzWmqlfUIz' List<NVPair> headerList = new ArrayList<NVPair>() headerList.add(new NVPair("Content-Type", "application/json")) headerList.add(HTTPPluginControl.getHTTPUtilities().basicAuthorizationHeader(uuid, key)) NVPair[] headers = headerList.toArray() // Invoke randomly chosen action in randomly chosen namespace. HTTPResponse result = request.POST("https://10.10.10.10/api/v1/namespaces/stub_user_" + randUserNum + "/actions/noopThroughput"+randActionNum+"?blocking=true&result=true", params, headers) assertThat(result.statusCode, is(200)); }
Run a test with multiple runtimes or different action memory setting.
Similarly you can create actions with multiple runtimes or with different memory setting.