Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Cleanup previous section

...

As of Hive 0.14 a registration mechanism has been introduced for native Hive SerDe's to bind .  This allows dynamic binding between a 'STORED AS' syntax naturally as a shorthand for keyword in place of a triplet of {SerDe, InputFormat, and OutputFormat} specification, in CreateTable statement statements.

The following mappings have been added through this registration mechanism:

SyntaxEquivalent

STORED AS AVRO/

STORED AS AVROFILE

ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
  STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'

STORED AS ORC/

STORED AS ORCFILE

ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
  STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'

STORED AS PARQUET/

STORED AS PARQUETFILE

ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
  STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
STORED AS RCFILE
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.RCFileInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.RCFileOutputFormat'
STORED AS TEXTFILE
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'

To add a new native SerDe with STORED AS syntaxkeyword, follow these steps:

  1. Create a storage format descriptor class extending from AbstractStorageFormatDescriptor.java that returns a stored as keyword to {, and name of InputFormat, OutputFormat, and SerDe } triplet mappingclasses.

  2. Add the name of the storage format descriptor class to StorageFormatDescriptor registration file.

...