Page History

...

In the existing setup, host affinity in yarn is accomplished via two phases:

A. Job model generation phase which generates an optimal processor to task assignment.
B. ContainerAllocator phase which requests resources from the cluster manager for each container.
Existing model
Zookeeper is used in standalone uses zookeeper for coordination amongst the processors in of a groupstream application. Amongst all the available processors of a stream application, a single processor will be chosen as a leader. The leader will generate the JobModel and propagate the JobModel to all other processors in the group. Distributed barrier in zookeeper will be used to block processing until the recent JobModel is picked by all the processors in the group.

...