Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Definition

Type that determines how def~commit will be stored and handled relative to its application to the target def~dataset.data will be laid out as file and stored, inside a def~table.


Image Added#todo verify


Following table summarizes the trade-offs between these two storage table types

Trade-offdef~copy-on-write (COW)def~merge-on-read (MOR)
Data LatencyHigherLower
Update cost (I/O)Higher (rewrite entire def~dataset def~table parquet)Lower (append to `delta log`)Parquet File SizeSmaller (high update(I/0) cost)Larger (low update cost)
Write AmplificationHigherLower (depending on compaction strategy to the def~dataset parquet def~table parquet)
Query/Read AmplificationLower/ZeroHigher (merging base and deltas on the fly)

Related concepts

  1. def~commit
  2. Commit List
  3. def~merge-on-read (MOR)
  4. def~copy-on-write (COW)
  5. def~timeline-instant

Status (draft)