...
This document proposes the introduction of the Executor
interface. This interface will abstract the execution/submission logic from the environments. The executor will take as argument an abstract representation of the user’s job and a configuration object containing the parameters that are relevant for that specific execution of that specific job and, for now, return a JobExecutionResult
, as shown below:
...
JobExecutionResult execute(Pipeline pipeline, Configuration executionConfig) throws Exception;
}
This signature is similar to that of the execute()
method of the ExecutionEnvironment
and StreamExecutionEnvironment
with the difference that it takes a Pipeline
as an argument. This is not coincidental.
...
Flink will use Service Discovery to discover the available factories. Then, based on the provided Configuration
, it will decide which executor to use. This configuration object will contain all the relevant parameters needed to instantiate the required Executor.
In terms of code, the above means that, for example, the execute()
method of the ExecutionEnvironment
will become a concrete method with the following body:
...