Versions Compared

Old Version 45

changes.mady.by.user Nishith Agarwal

Saved on

compared with

New Version 46

changes.mady.by.user Nishith Agarwal

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For CRDTs based concurrency management, with updates applied out of order, we want to generate the incremental pull snapshot by applying the same rules of CRDT so that no updates are lost for downstream consumers and they always see consistent state of the data. NOTE, for the OverwriteWithLatestRecord implementation is not a CRDT and special handling is needed for this as pointed below. 

This would mean that Spark, Presto and Hive MOR incremental pull have to implement the same semantics of ordering updates like in a RealtimeView and in compaction. 

...