Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Description: Currently, recovery upon failure backtracks until the data sources. This will add the option to checkpoint intermediate DataSets and backtrack from these checkpoints.

Associated JIRA:

...

Integrate with Zeppelin

Description: 

Associated JIRA:

Expected: 

Integrate with Tez

Description: Enable Flink programs to run on Tez rather than using Flink's network stack. For certain use cases, this will give the option of running Flink programs with the resource elasticity that Tez provides.

Associated JIRA:

Expected: First version supporting a subset of Flink API in Q1 2015

Integrate with Samoa

Description: 

Associated JIRA:

Expected: 

Semantic annotations for optimization

Description: A lot of optimizations are not possible in Flink, because the optimizer does not know what is happening inside user-defined functions. By adding semantic information for user functions which tells the optimizer how a function behaves, some of these limitations can be overcome.

Associated JIRA:

Expected: Q1 2015

Plan choice hints

DescriptionQuery optimizers are kind of black boxes and usually do a good job in finding efficient executions. However, in some cases the user/developer knows better and wants guide the optimizer or help to find a better plan. Flink’s optimizer offers several hints which are not well exposed in the API. Also documentation for how write well optimizable programs need to be improved.

Associated JIRA:

Expected: Q2 2015

Improved statistics for the optimizer

Description: 

Associated JIRA:

Expected: 

Use off-heap memory

Description: 

Associated JIRA:

Expected: 

Dynamic memory allocation

Description: 

Associated JIRA:

Expected: 



...