THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
No Format |
---|
hive -e "CREATE EXTERNAL TABLE IF NOT EXISTS hive_table_name (column_1 datatype_1......column_N datatype_N) PARTITIONED BY (partition_col_1 datatype_1 ....col_P datatype_P) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS INPUTFORMAT \"com.hadoop.mapred.DeprecatedLzoTextInputFormat\" OUTPUTFORMAT \"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\ ";" |
Note: The double quotes have to be escaped so that the 'hive -e
' command works correctly.
See CREATE TABLE and Hive CLI for information about command syntax.
Hive Queries
Option 1: Directly Create LZO Files
- Directly create LZO files as the output of the Hive query.
- Use
lzop
command utility or your custom Java to generate.lzo.index
for the.lzo
files.
...
No Format |
---|
hive -e "SET mapreduce.output.fileoutputformat.compress.codec=com.hadoop.compression.lzo.LzopCodec;SET hive.exec.compress.output=true;SET mapreduce.output.fileoutputformat.compress=true;<query-string>" |
Option 2: Write Custom Java to Create LZO Files
- Create text files as the output of the Hive query.
- Write custom Java code to
- convert Hive query generated text files to
.lzo
files - generate
.lzo.index
files for the.lzo
files generated above
- convert Hive query generated text files to
...