Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Definition

Type that determines how def~commit will be stored and handled relative to its application to the target def~datasetdata will be laid out as file and stored, inside a def~table.

#todo verify

Following table summarizes the trade-offs between these two storage dataset types

Trade-offdef~copy-on-writedef~merge-on-read
Data LatencyHigherLower
Update cost (I/O)Higher (rewrite entire def~dataset def~table parquet)Lower (append to `delta log`)Parquet File SizeSmaller (high update(I/0) cost)Larger (low update cost)
Write AmplificationHigherLower (depending on compaction strategy to the def~dataset def~table parquet)

Related concepts

  1. def~commit
  2. Commit List
  3. def~merge-on-read
  4. def~copy-on-write
  5. def~timeline-instant

Status (draft)