...
Column Type | CSV Format | Object Format | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NULL value in the field | public static final String NULL_FIELD = "NULL"; | java null | ||||||||||||||||
ARRAY |
Refer https://issues.apache.org/jira/browse/SQOOP-1771 for more details | java Object[] | ||||||||||||||||
BINARY | byte array enclosed in quotes and encoded with ISO-8859-1 charset | java byte[] | ||||||||||||||||
BIT | true, TRUE, 1 false, FALSE, 0 ( not encoded in quotes ) Unsupported values should throw an exception | java boolean | ||||||||||||||||
DATE | YYYY-MM-DD ( no time) | org.joda.time.LocalDate | ||||||||||||||||
DATE_TIME | YYYY-MM-DD HH:MM:DD[.ZZZ][+/-XX] ( fraction and timezone are optional) Refer https://issues.apache.org/jira/browse/SQOOP-1846 for more details | org.joda.time. DateTime or org.joda.time. LocalDateTime (depends on timezone attribute ) | ||||||||||||||||
DECIMAL | BigDecimal (not encoded in quotes ),
| java BigDecimal scale and precision fields are handled via : | ||||||||||||||||
ENUM | Same as TEXT | java String | ||||||||||||||||
FIXED_POINT | Integer or Long, ( not encoded in quotes ) | java Integer or java Long ( depends on byteSize attribute and signed attribute) | ||||||||||||||||
FLOATING_POINT | Float or Double ( not encoded in quotes ) | java Double or java Float ( depends on byteSize attribute) | ||||||||||||||||
MAP |
| java.util.Map<Object, Object> | ||||||||||||||||
SET | same as ARRAY | java Object[] | ||||||||||||||||
TEXT | Entire string will be enclosed in single quotes and all bytes will be printed as they are will exception of following bytes
| java String | ||||||||||||||||
TIME | HH:MM:DD[.ZZZ] ( fraction is optional ) 3 digit milli second support only for time | org.joda.time.LocalTime ( No Timezone) | ||||||||||||||||
UNKNOWN | same as BINARY | same as java byte[] |
...
CSVIntermediateDataFormat
Relevant JIRA : SQOOP-555 and SQOOP-1350
...
NOTE: It may not be obvious but the current IDF design expect every new implementation of it to expose the CSV an ObjectArray formats in addition to its native format.
JSONIntermediateDataFormat
Relevant JIRA: SQOOP-1901
Avro Intermediate Data Format
SqoopIDFUtils
It is a utility class in sqoop to aid connectors in encoding data into expected CSV format and object format and also parsing the CSV string back to the prescribed object format.
No Format |
---|
https://issues.apache.org/jira/browse/SQOOP-1813 |
...
Food for Thought.?
(Some of the below are some serious shortcomings of the current design as it exists)
...