This document is a collection of examples of Sqoop commands and arguments that are commonly used together. Its a good starting place for those overwhelmed by all the Sqoop configuration options and want to start exploring with something tried and true.

Exporting data from CSV file to Teradata:

Note: Teradata connector is required.

sqoop export --connect jdbc:teradata://terahost/DATABASE=MYDB  \    
--username myuser   --password mypass    \ 
--export-dir /user/hive/warehouse/mydb.db/data_stg   \ 
--table MYTABLE    --input-fields-terminated-by ','  

 

Merging data into Oracle with Oraoop:

sqoop export -Doraoop.export.merge=true -Doraoop.update.key.extra.columns="period_start_time"  \
-D mapred.map.max.attempts=1  \
--connect ... --username user --password pass --table table2 --export-dir /user/hive/warehouse/tables/mytable \
 --verbose -m 16 --update-key co_gid

 

Exporting a Hive table stored in a custom schema to PostgreSQL:

Sqoop does not have a notion of custom schemas. You need to specify the parameter --schema with a schema name if your table is stored in a different schema. Please note that the Hive schema must have the same name as the Postgres one. The --schema parameter must be separated from the rest of the parameters with an extra set of dashes (i.e. -- ) and the --schema parameter must come last.

sqoop export --connect jdbc:postgresql://postgresql.example.com/database --username sqoop --password sqoop  \
--table cities --export-dir cities  --  --schema us

 

Exporting HDFS dir to Netezza
sqoop export --direct --connect jdbc:netezza://NetezzaHost:5480/MyTable --username USER \ 
--password-file <pwd file location> --batch --export-dir <HDFS dir to export> --table NZTable \ 
--input-fields-terminated-by '\0001' --input-null-string '\\N' --input-null-non-string '\\N’ 

 

  • No labels