Page History

...

JobModel is the data model in samza that logically represents a samza job. The JobModel hierarchy is that samza jobs have one to many containers(ContainerModel), and each container has one to many tasks(TaskModel). Each data model contains relevant information, such as logical id, partition information, etc. In the standalone deployment model, the JobModel is stored in zookeeper. Coordinator stream(kafka topic) is used to store the JobModel in the yarn deployment model.

Existing Existing host affinity implementation of host affinity in yarn is accomplished through the following two phases:
A. ApplicationMaster(JobCoordinator) in yarn deployment model generates the Job model(optimal processor to task assignment) and persists the JobModel in coordinator stream(kafka topic) associated with the samza job.
B. ContainerAllocator phase(which happens after JobModel generation) schedules each stream processor to run on a physical host by coordinating with the underlying ClusterManager and orchestrates the execution of the stream processors.

...