...

Compatibility

1. In programming API: 

...

1. 1. the ApplicationRunner is a new interface, no compatibility issue
   2. JobRunner is going to be deprecated and replaced by LocalJobRunner and RemoteJobRunner. However, JobRunner is a Samza internal class. Hence, no interface compatibility issue.
   3. ThreadJobFactory and ProcessJobFactory will be deprecated. The LocalJobRunner is going to use StreamProcessor embedded library to replace those two classes.
2. In configuration: the configuration for ApplicationRunner is at a new scope: app.*. Hence, no change to the existing configuration
3. In launch script:
   1. New applications will use run-app.sh and run-local-app.sh for remote and local execution
   2. Old task-level application will continue to use run-job.sh and run-container.sh to submit and launch the local containers, where run-job.sh will be a wrapper for a single node application
4. In metrics: no change in the metric names and reporter interfaces

Deprecation

Existing JobRunner class is going to be deprecated, together with ThreadJobFactory/ProcessJobFactory, given that LocalJobRunner will use StreamProcessor (i.e. standalone Samza lib) to replace ThreadJobFactory/ProcessJobFactory.

Migration Plan

ApplicationRunner is currently designed for users writing new application. Existing Samza jobs using task-level API will need to do source code change to migrate to the new APIs. Hence, there is no auto-migration plan.

Rejected Alternatives

1. Using the same interface method run() in both RemoteApplicationRunner and LocalApplicationRunner: this causes confusion since run() in LocalApplicationRunner is blocking while in RemoteApplicationRunner is non-blocking.
2. Expose StreamProcessor to the user who wants to run the application locally: this will potentially causing the following issues:
   1. Divergence of programming model for Samza application running in YARN vs running locally
   2. Divergence of application launching model for Samza application running locally written in fluent API vs task-level API
   3. Exposure of multiple stages of jobs to the user who only cares about the end-to-end Samza application

...