Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: add table of contents
Table of Contents
minLevel2

How To Test Pig

This document covers how to test Pig. It is intended for Pig developers who need to know how to test their work. It can also be used by Pig users who wish to verify their instance of Pig.

...

Key

What it Does

Example

Required?

delimiter

Provides floatpostprocess with delimiter to use

'delimiter' => ':'

Only with flostpostprocess

execonly

This test will only be executed in specified mode; options are local and mapred

'execonly' => 'mapred'

No

expected_err_regex

Checks stderr error for the provided regular expression

'expected_err_regex' => "Out of bound access."

No

expected_out_regex

Checks stdout error for the provided regular expression

'expected_out_regex' => "A: {name: bytearray,age: bytearray,gpa: bytearray}"

No

floatpostprocess

Run floating point numbers through a post processor, since due to precision issues different runs of the same script will produce slightly different values. All floating point numbers are rounded to 3 decimal places. This must be used in conjunction with delimiter

'floatpostprocess' => 1

For outputs that include calculated floating point values.

ignore

Do not run this test, used when a test is failing but we don't want to remove it because it will be needed once the issue is fixed. A reason for ignoring the test should be given.

'ignore' => 'JIRA-19999'

No

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9c2f8c187d82f476-4ee16194-4b2f4e05-b89fb2f4-0e458e181ecf55835dfd6c1d"><ac:plain-text-body><![CDATA[

java_params

Values to be passed on the pig command line before other Pig parameters; useful for passing properties.

'java_params' => ['-Dpig.cachedbag.memusage=0']

No

]]></ac:plain-text-body></ac:structured-macro>

not_expected_err_regex

Checks that stderr does not match the provided regular expression

'not_expected_err_regex' => "ERROR"

No

not_expected_out_regex

Checks that stdout does not match the provided regular expression

'not_expected_out_regex' => "datafile"

No

notmq

Tells the test harness this is not a multi-query test; only necessary when a test has multiple store operators but should not be verified as if it were multi-query.

'notmq' => 1

No

num

Test number; must be unique in the test group

'num' => 1

Yes

pig

The Pig Latin script to run in the test

q#a = load ':INPATH:/dir/studenttab10k' as (name, age, gpa); store a into ':OUTPATH:';#

Yes

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="4e9b8d41b772ced9-93e5c08e-45684488-8da98f4f-8a5de74643f3c611478f4dd9"><ac:plain-text-body><![CDATA[

pig_params

Command line arguments to pass to pig when running this test.

'pig_params' => ['-p', qq(fname='studenttab10k')]

No

]]></ac:plain-text-body></ac:structured-macro>

rc

Expected return code

'rc' => 0

No

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="a9397c01515a4a92-0df7ebca-45cf4952-8b8a9884-b9571399f849acf9def6204b"><ac:plain-text-body><![CDATA[

sortArgs

Arguments to pass to the Unix sort utility. When these are given, sort will be called before data is sorted for comparison with the expected results.

'sortArgs' => ['-t', ' ', '+0', '-1']

Only when job output should be sorted

]]></ac:plain-text-body></ac:structured-macro>

verify_pig_script

Alternate Pig Latin script to use to generate the expected results

'verify_pig_script' => q\A = load ':INPATH:/singlefile/studenttab10k' as (name, age, gpa); store A into ':OUTPATH:';\,

No

...