THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block | ||
---|---|---|
| ||
// '+' denotes addition, '-' denotes deletion. public interface TaskNameGrouper { + @Deprecated Set<ContainerModel> group(Set<TaskModel> tasks); + @Deprecated default Set<ContainerModel> group(Set<TaskModel> tasks, List<String> containersIds) { return group(tasks); } + Set<ContainerModel> group(Set<TaskModel> taskModels, Map<String, LocationId> taskLocality, Map<String, LocationId> processorLocality); } + @Deprecated public interface BalancingTaskNameGrouper extends TaskNameGrouper { + @Deprecated Set<ContainerModel> balance(Set<TaskModel> tasks, LocalityManager localityManager); } public class ContainerModel { - @Deprecated - private final int containerId; private final String processorId; private final Map<TaskName, TaskModel> tasks; + // New field added denoting the physical locationId. + private final String locationId; } +public interface LocationIdProvider { + // In case of containerized environments, LocationId is a combination of multiple fields (sliceId, containerId, hostname) instead of simple physical hostname, + // Using a class to represent that, rather than a primitive string. This will be provided by execution environment. + LocationIdString getLocationId(); } + public interface LocalityManager { // returns the processorId to LocationId mapping. + public Map<String, LocationId>String> readProcessorLocality(); // returns the taskName to LocationId mapping. + public Map<String, LocationId>String> readTaskLocality(); // writes the provided processordId to host mapping to underlying storage. + public boolean writeProcessorLocality(Map<String, LocationId>String> processorLocality); } |
For yarn, preferred host mapping in the coordinator stream will be used for locality of processors and tasks unchanged between successive generations. If tasks or processor added in a run, in yarn it will be assigned to any new host.
...