Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Lack of deterministic termination detection and checkpoint support for the DataStream iteration.
  2. Lack of the support for multiple inputs, arbitrary outputs and nested iteration for both iteration APIs.
  3. Lack of asynchronous iteration support for the DataSet iteration. 
  4. The current current DataSet iteration by default provides a "for each round" semantics, namely users only need to specify the computation logic in each iteration, and the framework would executes the subgraph multiple times until convergence. To cooperate with the semantics, the DataSet iteration will replay the 

Overall Design

To reduce the development and maintenance overhead, it would be preferred to have a unified implementation for different types of iterations. In fact, the different iteration types shares the same requirements in runtime implementation:

...