This page describes a proposed Samza Enhancement Proposal (SEP) process for proposing a major change to Apache Samza.

Why SEP ?

The purpose of SEP is to collate and document all planned major enhancement to Apache Samza. JIRA is still the tool for tracking bugs and progress, SEP give an accessible overview of the design proposal, discussions and final acceptance status of the proposal. SEP, in other words, is nothing but a central location for all design documents in Apache Samza.

...

Hopefully we can make these proportional in effort to their magnitude — small changes should just need a couple brief paragraphs, whereas large changes need detailed design discussions. This process also isn't meant to discourage incompatible changes — proposing an incompatible change is totally legitimate. Sometimes we will have made a mistake and the best path forward is a clean break that cleans things up and gives us a good foundation going forward. Rather this is intended to avoid accidentally introducing half thought-out interfaces and protocols that cause needless heartburn when changed. Likewise the definition of "compatible" is itself squishy: small details like which errors are thrown when are clearly part of the contract but may need to change in some circumstances, likewise performance isn't part of the public contract but dramatic changes may break use cases. So we just need to use good judgement about how big the impact of an incompatibility will be and how big the payoff is.

When to provide a SEP?

Any of the following should be considered a major change:

...

Not all compatibility commitments are the same. We need to spend significantly more time on public APIs and samza-core api behavior as these break code for users. They cause people to rebuild code and lead to compatibility issues in large multi-dependency projects (which end up requiring multiple incompatible versions). Configuration, monitoring, and command line tools can be faster and looser — changes here will break monitoring dashboards and require a bit of care during upgrades but aren't a huge burden. For the most part monitoring, command line tool changes, and configs are added with new features so these can be done with a single SEP.

What should be included in SEP?

A typical SEP should include the following sections:

• Problem: Describe the problem to be solved
• Motivation: Why the problem should be solved
• Proposed Changes: Describe the changes you want to do. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences, depending on the scope of the change.
• New or changed public interfaces, if any: Impact to any of the "compatibility commitments" described above. We want to call these out in particular so everyone thinks about them.
• Implementation / Test Plan (if applicable): Some large projects may consists of multiple sub-projects and requires more planning for implementation and testing. It may be prudent to call them out in the proposal.
• Migration Plan and Compatibility: If this feature requires additional support for a no-downtime upgrade describe how that will work. If a no-downtime upgrade for users cannot be support, describe the migration plan for the users.
• Rejected Alternatives: What are the other alternatives you considered and why are they worse? The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered.

Who should initiate and drive a SEP?

Anyone can initiate a SEP but you shouldn't do it unless you have an intention of getting the work done to implement it (otherwise it is silly).

What is the criteria to accept / reject a SEP?

We will be following the Apache voting guideline to drive our decisions regarding an SEP. Decisions will be made on the [VOTE] mailing thread for a particular SEP. All community members are encouraged to participate by replying to the VOTE mailing thread. The votes can have the following meaning:

...

The criteria for acceptance of a SEP is lazy majority.

Process

Here is the process for making a SEP:
1. Create a page which is a child of this one. Take the next available SEP number and give your proposal a descriptive heading. e.g. "SEP 73: Samza Standalone with ZK Coordination". If you don't have the necessary permissions for creating a new page, please ask on the development mailing list (dev@samza.apache.org).
2. Fill in the sections as described above
3. Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] SEP-{your SEP number} {your SEP heading} The discussion should happen on the mailing list and *not* on the wiki since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making.
4. Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority.
5. **Please** update the SEP wiki page, and the index below, to reflect the current stage of the SEP after a vote. This acts as the permanent record indicating the result of the SEP (e.g., Accepted or Rejected). Also report the result of the SEP vote to the voting thread on the mailing list so the conclusion is clear.

SEP Number

Next SEP Number: 1

Use this number as the identifier for your FLIP and increment this value.

SEP Index

| SEP | Status | Link to Discussion Thread |
|:---:|:------:|:--------------------------|
| 1 | DISCUSS | Link |

...