You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

This page describes a proposed Hudi Improvement Proposal (HIP) process for proposing a major change to Hudi (based largely on KIP).

To create your own HIP, go to the HIP Template and copy it as follows.


Be sure, to enter the HIP number at the top of the copied page before saving, to generate the correct url for the page.

When does a feature require a HIP ?

Whenever a feature is considered to be causing a “major change” in the project, such a feature requires a HIP. Any of the following can be considered a major change:

  • Any new component, module or code that introduces a new concept into the project or alters the behavior of an existing one

  • Any large code refactor to address general code reusability and structure. There is no strong definition for “Large” and whether or not the refactor requires a HIP can be discussed on the @dev mailing list.

  • Any change that impacts the underlying file format. More specifically changes to the HoodieLogFormat.

  • Any changes to the index of the dataset

    • HoodieBloomIndex and associated classes

    • HbaseIndex

  • Any change that impacts the public interfaces of the project listed below

    • Spark datasource usage of Hudi

    • HoodieWriteClient or HoodieReadClient

    • HoodieDeltaStreamer

Not all HIP’s require the same effort and detail. For critical changes such as the File Format and Index, we need to deeply discuss the trade-offs of making such a change and how it impacts current and new users. Any changes to such components affect the correctness of a dataset (backwards and forward versions supported). Other changes such as code refactor might require more details around abstractions but as long as there is good test coverage, a migration plan section can be avoided. It may happen that you are making a bunch of changes across many components to enable an already existing feature. For example, introducing a new config along with reporting metrics, enhancing a tool and also improving documentation and the onboarding experience. If all of these changes are linked to a general feature/idea, these can be grouped together under a single HIP.

Who should initiate the HIP ?

Anyone can initiate a HIP. Please note that if you are unsure of whether a feature already exists or if there is a plan already to implement a similar one, always start a discussion thread on the dev mailing list before initiating a HIP. This will help everyone get the right context and optimize everyone’s usage of time.

How do I initiate a HIP ?

Here is the process for making a HIP:

  1. Start a discussion thread on the Apache Hudi dev mailing list. The subject of the discussion thread can be DISCUSS [proposed idea] to create a HIP. Use this discussion thread to get an agreement from people on the mailing list that your proposed idea necessitates a HIP.

  2. Take a HIP number (at the bottom of this page) and use the template above to create a new HIP and fill up all the applicable sections. Update the discussion thread’s title to DISCUSS HIP [HIP number] [HIP title].

  3. Update the HIP number (increment by 1) at the bottom of this page.

  4. Once the proposal is finalized, add at least 50% of the PPMC members as approvers of your document. You are free to add any number of dev members to your reviewers list.

  5. Once all the questions and issues are addressed and the HIP approved by the approvers, this HIP can be moved to cWiki as the first final draft of the HIP.

HIP round-up

Next HIP Number: 3

Use this number as the identifier for your HIP and increment this value by 1

  • No labels