Excerpt |
---|
Contains information describing how to test Trafodion using the Trafodion test libraries and automated build tests. |
Note: A recent pull request, https://github.com/apache/trafodion/pull/869, changed the name of $MY_SQROOT to $TRAF_HOME. This page has been changed accordingly. If you are on an older code base, use $MY_SQROOT instead of $TRAF_HOME.
...
Table of Contents | ||||
---|---|---|---|---|
|
...
If adding a new feature, then check that it is either covered by an existing regression test or add a new test to an existing test suite.
Test Suites
Location: core/sql/regress
Directory | Usage |
---|---|
charsets | Tests Character Sets. |
compGeneral | Compiler test suite; tests optimizer-specific features. |
core | Tests a subset/sample of all features from all the test suites. |
executor | Tests the SQL Executor. |
fullstack2 | Similar to core but a very limited subset. |
hive | Tests HDFS access to Hive tables. |
newregr | Unused/Saved repository for some unpublished features. These are not run. |
qat | Tests basic DDL and DML syntax. |
privs1 | Privilege tests part 1 - authorization setup, utilities, misc |
privs2 | Privilege tests part 2 - grants and revokes |
seabase | Tests JNI interface to HBase. |
tools | Regression driver scripts and general regression scripts. |
udr | Tests the User Defined Routines (UDR) and TMUDF functionality. |
Check Test Results
On completion, the test run prints out a test summary. All tests should pass, or pass with known differences.
Test results are written to the runregr-sb.log
file in each component’s directory. Therefore, you can check the test results after the fact as follows:
Code Block | ||
---|---|---|
| ||
cd $MY$TRAF_SQROOTHOME/rundir grep FAIL */runregr-sb.log |
...
Do the following:
Code Block | ||
---|---|---|
| ||
cd $MY$TRAF_SQROOTHOME . ./sqenv.sh cd $MY$TRAF_SQROOTHOME/../sql/regress tools/runallsb |
Running an Individual Test Suite
Note : You have to run the prerequisite core test core/TEST000 before running any individual test. Note that when you run the full suite of tests (as above) , this is the first one that gets run.
Code Block | ||||
---|---|---|---|---|
| ||||
#run core/TEST000 if you haven't run it once already cd $MY_SQROOT $TRAF_HOME/../sql/regress/core export rundir=$TRAF_HOME/rundir export scriptsdir=$TRAF_HOME/../sql/regress mkdir $rundir/core #run the test cd $rundir/core $scriptsdir/core/runregr -sb TEST000 #run a test suite eg charsets cd $TRAF_HOME . ./sqenv.sh cd $MY$TRAF_SQROOTHOME/../sql/regress tools/runallsb charsets |
Running an Individual
...
Test
If You’ve Already Run the Test Suite
...
Code Block | ||
---|---|---|
| ||
cd $MY$TRAF_SQROOTHOME/../sql/regress/<suite> # You can add the following two exports to .bashrc or .profile for convenience export rundir=$MY$TRAF_SQROOTHOME/rundir export scriptsdir=$MY$TRAF_SQROOTHOME/../sql/regress # run the test cd $rundir/<suite> $scriptsdir/<suite>/runregr -sb <test> |
Code Block | ||||
---|---|---|---|---|
| ||||
cd $rundir/executor $scriptsdir/executor/runregr -sb TEST130 |
...
If you have not run any regression suites so far, then you will not have the required sub directories set up. You manually create them for each suite you want to run. You will also need to run core/TEST000 before you run the test you want to run.
Code Block | ||
---|---|---|
| ||
cd $MY$TRAF_SQROOTHOME/../sql/regress/<suite> # You can add the following two exports to .bashrc or .profile for convenience export rundir=$MY$TRAF_SQROOTHOME/rundir export scriptsdir=$MY$TRAF_SQROOTHOME/../sql/regress mkdir $rundir cd $rundir # <suitename> should match the name of each directory in $scriptsdir mkdir <suitename> #run core/TEST000 if you haven't run it once already cd $TRAF_HOME/../sql/regress/core export rundir=$TRAF_HOME/rundir export scriptsdir=$TRAF_HOME/../sql/regress mkdir $rundir/core #run the test cd $rundir/core $scriptsdir/core/runregr -sb TEST000 # run the test cd $rundir/<suite> $scriptsdir/<suite>/runregr -sb <test> |
...
Introducing a wait in the test will wait forever until you enter a character. This is another way to make the test pause to attach the debugger to the sqlci process.
Modify an Existing Test
If you would like to add coverage for your new change, you can modify an existing test.
Run the test after your modifications. If you are satisfied with your results, you need to modify the EXPECTED<test number>
file to reflect your new change. The standard way to do it is to copy the LOG<test number>
file to EXPECTED<test number>
file. See Modify Tests for more information.
Database Connectivity Services (DCS)
The DCS test suite is organized per the Maven standard.
JDBC T4 Tests
The code is written in Java, and is built and unit tested using Maven. The test suite organization and use follow Maven standards.
Instructions for setting up and running the test can be found in source tree at dcs/src/test/jdbc_test
.
If using a the local_hadoop environment, use the swjdbc script to run the tests.
ODBC Tests
The code is written for the Python 2.7 unittest framework.
It is run via the Testr
and Tox
.
Code Block | ||
---|---|---|
| ||
cd dcs/src/test/pytests ./config.sh -d <host>t:<port> -t # Location of your Linux ODBC driver tar file tox -e py27 |
Further instructions for setting up and running the test can be found in source tree at dcs/src/test/pytests
.
If using the local_hadoop environment, use the swpyodbc script to run the tests.
Functional Tests
Phoenix
The Phoenix tests provides basic functional tests for Trafodion. These tests were originally adapted from their counterpart atsalesforce.com
.
The tests are executed using Maven with a Python wrapper. You can run them the same way on your own workstation instance just like the way Jenkins runs them. Do the following:
- Prior to running Phoenix tests, you need to bring up your Trafodion instance and DCS. You need to configure at least 2-4 servers for DCS. The tests need at least two mxosrvrs as they make two connections at any given time. We recommend configuring DCS with four mxosrvrs since we have seen situations that mxosrvrs do not get released in time for the next connection if there are only two mxosrvrs.
Run the Phoenix tests from source tree.
Code Block language bash cd tests/phx phoenix_test.py --target=<host>:<port> --user=dontcare --pw=dontcare --targettype=TR --javahome=<jdk> --jdbccp=<jdir>/jdbcT4.jar
<host>
: Your workstation name or IP address.<port>
: your DCS master port number.<jdk>
: the directory containing thejdk1.7.0_21_64
or later version of the JDK.<jdir>
: the directory containing your JDBC T4 jar file. (export/lib
if you downloaded a Trafodion binary package.)
The source code can be found in
phoenix_test/src/test/java/com/trafodion/phoenix/end2end
. These are JDBC tests written in Java.Analyze the results. The test results can be found in
phoenix_test/target/surefire-reports
. If there are any failures, they would come with file names and line numbers.
Anchor test-automated-tests test-automated-tests
Automated Tests
Automated tests take several hours to complete from when your pull-request was approved by a committer or updated with a new commit.
Normally, the Traf-Jenkins user will post a message in the pull-request with a link to the results. You can also check the Jenkins server to see the status even before the tests are finished. Look in the Build History table for the build/test job that matches your pull-request. For example, the master branch tests are located at: https://jenkins.esgyn.com/job/Check-PR-master/
Reviewing Logs
There are two approaches to reviewing logs.
Approach 1
- The first two columns in build-job table are links to the specific sub-job. Click on the link to drill down.
- The console log of each job has a link to the log file directories (close to the top). Look for Detailed logs.
Approach 2
- Go to: http://traf-logstestlogs.esgyn.com/PullReq/
- Click on the number of the pull request. The next directory level is the build number. With multiple commits or re-tests, it is possible for a pull request to have multiple builds.
- Under the build number is a directory for each specific job.
- Example: http://traf-logstestlogs.esgyn.com/PullReq/18/35/regress-seabase-ahw2.2/
More Information
The check tests do not include all of the automated daily tests. If you (or another contributor) want, you can run additional tests on the pull request. Refer Automated Test Setup below for more information.
Anchor create-test-automated-test-setup create-test-automated-test-setup
Automated Test Setup
To be done
...