...

During the migration of the Iceberg Sink to the new Flink Sink V2 API we found that the current WithPreCommitTopology interface is not enough sufficient for our use-case. The WithPreCommitTopology interface looks like this:

...

• Every new feature should be added with a Supports<FeatureName> interface (similar to the Source API), like
  • SupportsCommiter
  • SupportsWriterState
  • SupportsPreWriteTopology
  • SupportsPreCommitToplogy
  • SupportsPostCommitTopology
• Keep the number of the inner interfaces/classes minimal, and expose them as a full fledged classes
  • Committer - no change here
  • CommitterInitContext
  • StatefulSinkWriter
  • WriterInitContext
• No redefining interface methods during interface inheritance - it would prevent future deprecation
• Minimal inheritance extension - for more flexibility in the future. Kept only
  • StatefulSinkWriter, CommittingSinkWriter - which inherits from SinkWriter
  • CommitterInitContext, WriterInitContext - which inherits from InitContext

...

Compatibility, Deprecation, and Migration Plan

Deprecation

We will deprecated deprecate the old interfaces, but provide a default implementation based on the new API to allow Sink developers to migrate to the new API conveniently.

PublicEvolving interfaces

...

These changes need to be deprecated and kept for at least 1 minor release (until 1.20 as per current plans at the point of writing this).

The proposed solution is to make the old interfaces extend the new ones.

...

As a first step, we create our own unit tests, but keep the original tests intact. This will allow as us to confirm, that the changes are backward compatible. The new unit tests will confirm that the new features are working.

...