timeline instant

Definition

An item in the `Hudi` ingestion processing timeline

Design details

At its core, Hudi maintains a timeline of all actions performed on the def~dataset at different instants of time that helps provide instantaneous views of the def~dataset, while also efficiently supporting retrieval of data in the order of arrival. A Hudi `timeline instant` consists of the following components

• instant action type: Type of action performed on the def~dataset
• instant time: typically a timestamp (e.g: 20190117010349), which monotonically increases in the order of action’s begin time.
• instant state: current state of the timeline instant

Design decisions

1. Hudi guarantees that the actions performed on the timeline are atomic & timeline consistent based on the instant time.

Key Instant action types performed include:

Any given instant can be in one of the following instant states:

Design decisions

1. #todo

Related concepts

1. def~dataset
2. instant state
3. instant action type
4. instant time
5. def~commit
6. file format

Status (draft)