Status
Current state: "Under Discussion"
Discussion thread: To be added
JIRA: <To be added>
Released: 1.14
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
The existing Flink ML library allows users to compose an Estimator/Transformer from a pipeline (i.e. linear sequence) of Estimator/Transformer, and each Estimator/Transformer has one input and one output.
The existing library API does not allow us to express a DAG of algorithms, where each algorithm logics (e.g. Transformer, Estimator) can have multiple inputs and multiple outputs.
In order to address the above use-case, we have come up with two possible designs with different flavor in usability.
Please find the design details below: