DRAFT
Table of Contents
Title: DFM support in Sqoop
JIRA : SQOOP-1168 and its sub tickets
The discussion also happened on the SQOOP-1168 and SQOOP-1804 and SQOOP-2025
Table of Contents |
---|
Summary
Provide a way for sqoop jobs to read and write a subset of records i.e transfer only new records added since the last transfer in case of immutable data sources and/or transfer the delta modifications across the entire data set in case of mutable data sources
...
Design Features Revision 2
( After some discussions on the https://issues.apache.org/jira/browse/SQOOP-1168, adding more details to resolved open questions)UPDATE: The discussion happened on the SQOOP-1168 and SQOOP-1804 and SQOOP-2025 JIRA items. The design doc will soon be updated with the final details.
NOTE : We will use the DFM term to broadly mean reading sub set of records and write them either my appending or reconciling/ overwriting with existing values/ deleting. It can mean any of the above or all as the connector implementation defines it.
...