Storage Formats

SerDes and Storage Formats

HCatalog uses Hive's SerDe class to serialize and deserialize data. SerDes are provided for RCFile, CSV text, JSON text, and SequenceFile and ORC formats. Check the Hive SerDe documentation for additional SerDes that might be included in new versions. For example, the Avro SerDe was added in Hive 0.9.1 and , the ORC file format was added in Hive 0.11.0, and Parquet was added in Hive 0.10.0 (plug-in) and Hive 0.13.0 (native).

Users can write SerDes for custom formats using these instructions in the Hive SerDe documentation:

• Hive How to Write Your Own SerDe in the Developer GuideSerDe - how to add a new SerDe in the Developer Guide
• Hive User Group Meeting August 2009 pages 64-70
• also see SerDe for details about input and output processing

...

See HCATALOG-436 for details.