Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
// '+' denotes addition, '-' denotes deletion.
public interface TaskNameGrouper {
  + @deprecated
  Set<ContainerModel> group(Set<TaskModel> tasks);

  + @deprecated
  default Set<ContainerModel> group(Set<TaskModel> tasks, List<String> containersIds) {
    return group(tasks);
  }
  +   Set<ContainerModel> group(Set<TaskModel> currentGenerationTaskModels, List<String> currentGenerationProcessorIds, Set<ContainerModel> previousGenerationContainerModels);
}

public interface BalancingTaskNameGrouper extends TaskNameGrouper {
  + @deprecated 
  Set<ContainerModel> balance(Set<TaskModel> tasks, LocalityManager localityManager);
}

public class ContainerModel {
  - @Deprecated
  - private final int containerId;
  private final String processorId;
  private final Map<TaskName, TaskModel> tasks;
  + // New field added denoting the physical hostnamelocationId.
  + private final String hostNamelocationId;
}
 
+public interface LocationIdProvider {
   LocationId getLocationId();
 }

+ public class LocalityInfo { 
+  // In case of containerized environments, LocationId is a combination of multiple fields (sliceId, containerId, hostname) instead of simple physical hostname,
+  // Using a class to represent that, rather than a primitive string. This will be provided by execution environment.
+  private Map<String, LocationId> taskLocality;
+  public Map<String, LocationId> getLocality();  
}

+ public interface LocalityManager {
   // returns processorId to host mapping.
  + public LocalityInfo readProcessorLocality();
  // writes the provided processordId to host mapping to underlying storage.
  + public boolean writeProcessorLocality(LocalityInfo localityInfo);
}

...

Compatibility, Deprecation, and Migration Plan

  • In version 0.15upcoming release, LocalityManager class will be changed into an interface. Any code which depends on LocalityManager class directly in open source should migrate to CoordinatorStreamBasedLocalityManager.
  • We are not altering the data storage format of the ContainerModel in coordinator stream for yarn deployment model.
  • ContainerId field in ContainerModel which is deprecated in samza 0.13 version will be removed in the version 0future release. 15. Open source users using containerId field from ContainerModel should migrate and use processorID field in ContainerModel.
  • All of the existing methods in TaskNameGrouper and BalancingTaskNameGrouper will be deprecated. 
  • It’s recommended that the users recompile their deployable after migrating to the samza version 0.15that has this feature.
  • Will add compatibility test to verify that deprecating/changing the TaskNameGrouper API changes does not alter the existing behaviors.

Rejected Alternatives

Approach 1

...