def~timeline

Definition

An item in the `Hudi` ingestion processing timeline

Design details

At its core, Hudi maintains a timeline of all actions performed on the def~table at different instants of time that helps provide instantaneous views of the def~table, while also efficiently supporting retrieval of data in the order of arrival. A Hudi `timeline instant` consists of the following components

• def~instant-action-type: Type of action performed on the def~table
• def~instant-time: typically a timestamp (e.g: 20190117010349), which monotonically increases in the order of action’s begin time.
• instant state: current state of the def~timeline

Design decisions

1. Hudi guarantees that the actions performed on the timeline are atomic & timeline consistent based on the instant time.

Key Instant action types performed include:

Any given instant can be in one of the following instant states:

Any given instant can be in one of the following states:

• REQUESTED - Denotes an action has been scheduled, but has not initiated
• INFLIGHT - Denotes that the action is currently being performed
• COMPLETED - Denotes completion of an action on the timeline

Design decisions

1. #todo

Related concepts

1. def~table
2. instant state
3. def~instant-action-type
4. def~instant-time
5. def~commit
6. file format

Status (draft)