Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Definition

An item in the `Hudi` ingestion processing timelineA log of def~instant-actions that are performed on a def~table, ordered by def~instant-time.

Design details


Excerpt

At its core, Hudi maintains a timeline of all

...

def~instant-action performed on the def~table at different instants of time that helps provide instantaneous views of the def~table, while also efficiently supporting retrieval of data in the order

...

Design decisions

...

in which it was written. The timeline is akin to a redo/transaction log, found in databases, and consists of a set of def~timeline-instants. Hudi guarantees that the actions performed on the timeline are atomic & timeline consistent based on the instant time. Timeline is implemented as a set of files under the `.hoodie` def~metadata-folder directly under the def~table-basepath. Specifically, while the most recent instants are maintained as individual files, the older instants are archived to the def~timeline-archival folder, to bound the number of files, listed by writers and queries.

...

 


Design decisions


Excerpt Include
def~instant-action-type
def~instant-action-type
nopaneltrue
Any given instant can be in one of the following instant states:

Excerpt Include
def~instant-state
def~instant-state
nopaneltrue

Design decisions

  1. #todo

Related concepts

  1. def~table
  2. instant state
  3. def~instant-action-type
  4. def~instant-time
  5. def~commit
  6. file format

Status (draft)