Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Unit tests are executed via JUnit. Currently, many "unit tests" are really end-to-end tests. We are in the process of changing this so that all of end-to-end tests will be run by the e2e harness (see below). See PigTestProposal for details.

Running all unit tests

To run the unit tests do ant test in the top level Pig directory. Currently this takes 8 hours to run. We intend to drive this to under five minutes. Until this is done it is not expected that contributors will run all these tests before submitting their patch.

Running pre-commit tests

You can also run ant test-commit. This runs true unit tests plus a few very basic end-to-end tests to assure nothing fundamental has been broken. We ask that committers run this before committing patches, and that contributors run it before uploading their patch.

...

Key

What it Does

Example

Required?

delimiter

Provides floatpostprocess with delimiter to use

'delimiter' => ':'

Only with flostpostprocess

execonly

This test will only be executed in specified mode; options are local and mapred

'execonly' => 'mapred'

No

expected_err_regex

Checks stderr error for the provided regular expression

'expected_err_regex' => "Out of bound access."

No

expected_out_regex

Checks stdout error for the provided regular expression

'expected_out_regex' => "A: {name: bytearray,age: bytearray,gpa: bytearray}"

No

floatpostprocess

Run floating point numbers through a post processor, since due to precision issues different runs of the same script will produce slightly different values. All floating point numbers are rounded to 3 decimal places. This must be used in conjunction with delimiter

'floatpostprocess' => 1

For outputs that include calculated floating point values.

ignore

Do not run this test, used when a test is failing but we don't want to remove it because it will be needed once the issue is fixed. A reason for ignoring the test should be given.

'ignore' => 'JIRA-19999'

No

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="ef98a604b000ab5e-335dadf7-4a8945e2-89e9b810-45c5fa63233a420dbde0f936"><ac:plain-text-body><![CDATA[

java_params

Values to be passed on the pig command line before other Pig parameters; useful for passing properties.

'java_params' => ['-Dpig.cachedbag.memusage=0']

No

]]></ac:plain-text-body></ac:structured-macro>

not_expected_err_regex

Checks that stderr does not match the provided regular expression

'not_expected_err_regex' => "ERROR"

No

not_expected_out_regex

Checks that stdout does not match the provided regular expression

'not_expected_out_regex' => "datafile"

No

notmq

Tells the test harness this is not a multi-query test; only necessary when a test has multiple store operators but should not be verified as if it were multi-query.

'notmq' => 1

No

num

Test number; must be unique in the test group

'num' => 1

Yes

pig

The Pig Latin script to run in the test

q#a = load ':INPATH:/dir/studenttab10k' as (name, age, gpa); store a into ':OUTPATH:';#

Yes

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="fde4ad722523b1f4-2e6a09e7-46e3411e-ba8f9841-5923cc2d243763accbf78dae"><ac:plain-text-body><![CDATA[

pig_params

Command line arguments to pass to pig when running this test.

'pig_params' => ['-p', qq(fname='studenttab10k')]

No

]]></ac:plain-text-body></ac:structured-macro>

rc

Expected return code

'rc' => 0

No

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="b276e952ecd42edc-9af527c8-4baf44a9-8f9facf8-e2e82c1b594c3d04f945c7ae"><ac:plain-text-body><![CDATA[

sortArgs

Arguments to pass to the Unix sort utility. When these are given, sort will be called before data is sorted for comparison with the expected results.

'sortArgs' => ['-t', ' ', '+0', '-1']

Only when job output should be sorted

]]></ac:plain-text-body></ac:structured-macro>

verify_pig_script

Alternate Pig Latin script to use to generate the expected results

'verify_pig_script' => q\A = load ':INPATH:/singlefile/studenttab10k' as (name, age, gpa); store A into ':OUTPATH:';\,

No

...