Status
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
As one of the release 2.0 efforts, the release managers were discussing about our API life cycle policies. There have been FLIP-196 and FLIP-197 that are relevant to this topic. These two FLIPs defined the stability guarantees of the programming APIs with various different stability annotations, and the promotion process. A recap of the conclusion is following:
Stability:@Internal
API: can change between major/minor/patch releases.@Experimental
API: can change between major/minor/patch releases.@PublicEvolving
API: can change between major/minor releases.@Public
API: can only change between major releases.
Promotion:
An @Experimental
API should be promoted to @PublicEvolving
after two releases, and a @PublicEvolving
API should be promoted to @Public
API after two releases, unless there is a documented reason not to do so.
One thing did not mention in these two FLIPs is the API deprecation process, which is in fact critical and fundamental to how the stability guarantees to the end user because the sensible API stability is all about removing existing APIs. For example, if we want to change a method ResultFoo foo(ArgumentFoo arg)
to ResultBar bar(ArgumentBar arg)
, there are two ways:
- Mark method
foo
as deprecated and add the new methodbar
. At some point later, remove the methodfoo
. - Simply change the API in place, that basically means removing method foo and add method
bar
at the same time, i.e. replace methodfoo
with methodbar
.
In the first option, users are given a period with stability guarantee to migrate from foo
to bar
. For the second option, this migration period is effectively zero. A zero migration period is problematic because end users may need a feature/bug fix from a new Flink version, but they cannot upgrade right away due to some backwards compatible changes, even though these changes perfectly comply with the API stability guarantees defined above. With a migration period, users will be able to better arrange their migration timing, which makes the upgrade more smooth. Therefore, ideally the migration period should never be 0, even for the @Experimental APIs.
As you may see, the migration period is fundamental to the API stability guarantees for the end users. And it is essentially how long a deprecated API can be removed from the source code. This FLIP proposes an API deprecation process for Flink.
Public Interfaces
- Always add a "Since X.X.X" comment to indicate when was a class / interface / method marked as deprecated. So we can easily track when should the deprecated code be cleaned up in the deprecated list generated by Java doc. We should use ArchUnit to enforce this rule is possible.
- In the release notes, add a section about the deprecation to notify the users about the source code removal in advance.
In addition, a @Public interface can only be marked as deprecated if
- There are one or more @Public interfaces that serve as replacement of its functionality, or
- The functionality of the @Public interface is going to be removed completely without any full or partial replacement.
Proposed Changes
The migration periods
API Annotation | Migration Period | DESCRIPTION with Examples |
---|---|---|
Experimental | At least 1 patch release for the affected minor release. | Deprecation process:
User migration path:
NOTES:
|
PublicEvolving | At least 1 minor release. | Deprecation Process:
User migration path:
NOTES:
|
Public | At least 2 minor releases. | Deprecation Process:
User migration path:
NOTES:
|
Note that the same requirement for API change is still needed for all the Experimental/PublicEvolving/Public API changes.
Compatibility, Deprecation, and Migration Plan
- The changed proposed in the FLIP are all about processes and conventions of the source code deprecation and removal, which are not clearly defined at the moment. So there should be no compatibility, deprecation or migration issue.
Test Plan
We can potentially add ArchUnit tests to ensure the deprecation information is provided correctly.
Rejected Alternatives
N/A