Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

PigStorage also load/store complex data type: tuple, bag, map. Here are formats:

  • Tuple: enclosed by (), items separated by ",". Eg: (item1,item2,item3). Empty tuple is valid. Eg: ()
  • Bag: enclosed by {}, tuples separated by ",". Eg: {code}{(tuple1),(tuple2),(tuple3)}{code}. Empty bag is valid. Eg: {}
  • Map: enclosed by [], items separated by ",", key and value seperated by "#". Eg: [key1#value1,key2#value2]. Empty map is valid. Eg: []

If load statement specify a schema, Pig will convert the complex type according to schema. If conversion fail, the affected item will be null.

Here are more examples:

Code Block
a = load '1.txt' as (a0:{t:(m:map[int],d:double)});

{([foo#1,bar#2],34.0),([white#3,yellow#4],45.0)} : valid
{([foo#badint],baddouble)} : conversion fail for badint/baddouble, get {([foo#],)}
{} : valid, empty bag