Versions Compared

Old Version 7

changes.mady.by.user Veena Basavaraj

Saved on

compared with

New Version 8

changes.mady.by.user Veena Basavaraj

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

DRAFT

 

Table of Contents

 

Title: DFM support in  Sqoop

JIRA : SQOOP-1168 and its sub tickets

The discussion also happened on the SQOOP-1168 and SQOOP-1804 and SQOOP-2025

 

Table of Contents

 

Summary

Provide a way for sqoop jobs to read and write a subset of records i.e transfer only new records added since the last transfer in case of immutable data sources and/or  transfer the delta modifications across the entire data set in case of mutable data sources 

...

Design Features Revision 2 

( After some discussions on the https://issues.apache.org/jira/browse/SQOOP-1168, adding more details to resolved open questions)UPDATE: The discussion happened on the SQOOP-1168 and SQOOP-1804 and SQOOP-2025 JIRA items. The design doc will soon be updated with the final details.

NOTE : We will use the DFM  term to broadly mean reading sub set of records and write them either my appending or reconciling/ overwriting with existing values/ deleting. It can mean any of the above or all as the connector implementation defines it.

...