THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block | ||
---|---|---|
| ||
// '+' denotes addition, '-' denotes deletion. public interface TaskNameGrouper { + @deprecated Set<ContainerModel> group(Set<TaskModel> tasks); + @deprecated default Set<ContainerModel> group(Set<TaskModel> tasks, List<String> containersIds) { return group(tasks); } + Set<ContainerModel> group(Set<TaskModel> currentGenerationTaskModels, List<String> currentGenerationProcessorIds, Set<ContainerModel> previousGenerationContainerModels); } public interface BalancingTaskNameGrouper extends TaskNameGrouper { + @deprecated Set<ContainerModel> balance(Set<TaskModel> tasks, LocalityManager localityManager); } public class ContainerModel { - @Deprecated - private final int containerId; private final String processorId; private final Map<TaskName, TaskModel> tasks; + // New field added denoting the physical hostnamelocationId. + private final String hostNamelocationId; } +public interface LocationIdProvider { LocationId getLocationId(); } + public class LocalityInfo { + // In case of containerized environments, LocationId is a combination of multiple fields (sliceId, containerId, hostname) instead of simple physical hostname, + // Using a class to represent that, rather than a primitive string. This will be provided by execution environment. + private Map<String, LocationId> taskLocality; + public Map<String, LocationId> getLocality(); } + public interface LocalityManager { // returns processorId to host mapping. + public LocalityInfo readProcessorLocality(); // writes the provided processordId to host mapping to underlying storage. + public boolean writeProcessorLocality(LocalityInfo localityInfo); } |
...
Compatibility, Deprecation, and Migration Plan
- In version 0.15upcoming release, LocalityManager class will be changed into an interface. Any code which depends on LocalityManager class directly in open source should migrate to CoordinatorStreamBasedLocalityManager.
- We are not altering the data storage format of the ContainerModel in coordinator stream for yarn deployment model.
- ContainerId field in ContainerModel which is deprecated in samza 0.13 version will be removed in the version 0future release. 15. Open source users using containerId field from ContainerModel should migrate and use processorID field in ContainerModel.
- All of the existing methods in TaskNameGrouper and BalancingTaskNameGrouper will be deprecated.
- It’s recommended that the users recompile their deployable after migrating to the samza version 0.15that has this feature.
- Will add compatibility test to verify that deprecating/changing the TaskNameGrouper API changes does not alter the existing behaviors.
Rejected Alternatives
Approach 1
...