Dual streaming and batch engine

Description: Natively support both blocking and pipelined mode of execution for both batch (DataSet) and stream (DataStream) programs. Batch (DataSet) programs will be able to use a combination of blocking and pipelining. Stream (DataStream) programs will use pipelining. Interactive programs (programs that bring back results to the client) will use blocking.

Associated JIRA:

Expected: Q1 2015

Fine-grained fault tolerance for batch programs

Description: Currently, recovery upon failure backtracks until the data sources. This will add an option to checkpoint intermediate DataSets and backtrack from checkpoints.

Associated JIRA:

Expected: Q2 2015

Interactive programs

Description: Programs that are partially executed in the cluster and partially in the client, They consist of many small programs submitted by the driver program, with driver-side logic in-between.

Associated JIRA:

Expected: Q1 2015

Interactive Scala shell

Description: Be able to run Flink interactive programs from a Scala shell

Associated JIRA:

Expected: Q2/Q3 2015

Machine Learning library

Description: Be able to run Flink interactive programs from a Scala shell

Associated JIRA:

Expected: Q2/Q3 2015