THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Alert | ||||
|---|---|---|---|---|
| ||||
|
This page describes a proposed Hudi Improvement Proposal (HIP) process Request For Comments (RFC) process for proposing a major change to Hudi (based largely on KIP)or even just sharing designs/vision for the project to get early feedback.
To create your own HIPRFC, go to the HIP RFC Template and copy it as follows.
Be sure, to enter the HIP RFC number at the top of the copied page before saving, to generate the correct url for the page.
When does a feature require a
...
RFC ?
Whenever a feature is considered to be causing a “major change” in the project, such a feature requires a HIPRFC. Any of the following can be considered a major change:
Any new component, module or code that introduces a new concept into the project or alters the behavior of an existing one
Any large code refactor to address general code reusability and structure. There is no strong definition for “Large” and whether or not the refactor requires a HIP RFC can be discussed on the @dev mailing list.
Any change that impacts the underlying file format. More specifically changes to the HoodieLogFormat.
Any changes to the index of the dataset
HoodieBloomIndex and associated classes
HbaseIndex
Any change that impacts the public interfaces of the project listed below
Spark datasource usage of Hudi
HoodieWriteClient or HoodieReadClient
HoodieDeltaStreamer
- It can also be used to describe large direction shifts to the project (e.g: Flink support) or new green field ideas (e.g Hudi for ML flows)
Not all RFCs Not all HIP’s require the same effort and detail. For critical changes such as the File Format and Index, we need to deeply discuss the trade-offs of making such a change and how it impacts current and new users. Any changes to such components affect the correctness of a dataset (backwards and forward versions supported). Other changes such as code refactor might require more details around abstractions but as long as there is good test coverage, a migration plan section can be avoided. It may happen that you are making a bunch of changes across many components to enable an already existing feature. For example, introducing a new config along with reporting metrics, enhancing a tool and also improving documentation and the onboarding experience. If all of these changes are linked to a general feature/idea, these can be grouped together under a single HIPRFC.
Who should initiate the
...
RFC ?
Anyone can initiate a HIPRFC. Please note that if you are unsure of whether a feature already exists or if there is a plan already to implement a similar one, always start a discussion thread on the dev mailing list before initiating a HIPRFC. This will help everyone get the right context and optimize everyone’s usage of time.
How do I initiate a
...
RFC ?
Here is the process for making a HIPRFC:
Start a discussion thread on the Apache Hudi dev mailing list. The subject of the discussion thread can be DISCUSS [proposed idea] to create a HIPRFC. Use this discussion thread to get an agreement from people on the mailing list that your proposed idea necessitates a HIPRFC.
Take a HIP RFC number (at the bottom of this page) and use the template above to create a new HIP RFC and fill up all the applicable sections. Update/Create the discussion thread’s thread with title to "[DISCUSS HIP ] RFC [HIP RFC number] [HIP RFC title]."
Update the HIP RFC number (increment by 1) at the bottom of this page.
Once the proposal is finalized, add at least 50% 2 of the PPMC members as approvers of your document. You are free to add any number of dev members to your reviewers list.
Once Work through feedback until all the questions and issues are addressed and the HIP RFC approved by the approvers, this HIP can be moved to cWiki as the first final draft of the HIP.
...
RFC round-up
Next HIP RFC Number: 335
Use this number as the identifier for your HIP RFC and increment this value by 1
...