...
Running the e2e tests requires three things: a cluster to run them on, an old version of Pig to use to generate expected results, and perl and a cpan module IPC::Run (plus a few CPAN modules) on your client machine. The cluster can be quite small. Four machines will enough, one for a Name Node/Job Tracker and three for Data Node/Task TrackersA single machine will enough. Since performance is not the goal it is fine if these are virtual machinesthis is a virtual machine. If you do not have access to a cluster, see below for information on how to run the tests on EC2.For installing the
You will need the following CPAN modules:
- IPC::Run
- Parallel::ForkManager
- DBI
For help installing CPAN modules perl module, see cpan module install instructions .
...
Key | What it Does | Example | Required? | ||
---|---|---|---|---|---|
delimiter | Provides | | Only with | ||
execonly | This test will only be executed in specified mode; options are | | No | ||
expected_err_regex | Checks stderr error for the provided regular expression | | No | ||
expected_out_regex | Checks stdout error for the provided regular expression | | No | ||
floatpostprocess | Run floating point numbers through a post processor, since due to precision issues different runs of the same script will produce slightly different values. All floating point numbers are rounded to 3 decimal places. This must be used in conjunction with | | For outputs that include calculated floating point values. | ||
ignore | Do not run this test, used when a test is failing but we don't want to remove it because it will be needed once the issue is fixed. A reason for ignoring the test should be given. | | No | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="66b66127d81bc661-77089fb9-42724adc-8fc992ca-e9321e092634c2f95050e799"><ac:plain-text-body><![CDATA[ | java_params | Values to be passed on the | | No | ]]></ac:plain-text-body></ac:structured-macro> |
not_expected_err_regex | Checks that stderr does not match the provided regular expression | | No | ||
not_expected_out_regex | Checks that stdout does not match the provided regular expression | | No | ||
notmq | Tells the test harness this is not a multi-query test; only necessary when a test has multiple | | No | ||
num | Test number; must be unique in the test group | | Yes | ||
pig | The Pig Latin script to run in the test | | Yes | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="5e078217b6d144bd-e89654ba-43ad4db0-a49987a8-f30aab661c8d7fbc35be78e6"><ac:plain-text-body><![CDATA[ | pig_params | Command line arguments to pass to | | No | ]]></ac:plain-text-body></ac:structured-macro> |
rc | Expected return code | | No | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="b3b8c641722a1848-c870cdf1-49094b55-9a9ba480-730564a85d374391f5b250e7"><ac:plain-text-body><![CDATA[ | sortArgs | Arguments to pass to the Unix | | Only when job output should be sorted | ]]></ac:plain-text-body></ac:structured-macro> |
verify_pig_script | Alternate Pig Latin script to use to generate the expected results | | No |
...