Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The open()
method in RichFunction
requires a Configuration
instance as an argument, which is always passed as a new instance without any configuration parameters in AbstractUdfStreamOperator#open
. Thus, it is unnecessary to include this parameter in the open() method.
Public Interfaces
RichFunction#open() method:
- open(Configuration parameters)
Proposed Changes
The first change is to remove the parameter in RichFunction#open().
However, there are three implementations of RichFunction
that still utilize the empty configuration in their open()
method:
1. ContinuousFileMonitoringFunction#open
: This method uses the configuration to configure the FileInputFormat
.
2. OutputFormatSinkFunction#open
: This method uses the configuration to configure the OutputFormat
.
3. InputFormatSourceFunction#open
: This method uses the configuration to configure the InputFormat
.
To remove the dependence of In/OutputFormat
on the configuration parameter of the open()
method, there are several options available:
Option 1: Use a new empty configuration to configure the In/OutputFormat
in the open()
method.
Option 2: Remove the format configuration process for ContinuousFileMonitoringFunction#open, OutputFormatSinkFunction#open
, and InputFormatSourceFunction#open
.
Option 3: Add a new configure() method with no parameter to the In/OutputFormat
interface, and utilize it in ContinuousFileMonitoringFunction#open
, OutputFormatSinkFunction#open
, and InputFormatSourceFunction#open
.
Option 1 requires minimal modifications, and other suggestions are also welcomed.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
For the users that rely on the open() method, they will have to migrate to the no parameter variants. This migration shouldn't be difficult.
As the RichFunction abstract class is part of the public API we will deprecate the method 2 releases before removal.