Versions Compared

Old Version 37

changes.mady.by.user Shanthoosh Venkataraman

Saved on

compared with

New Version 38

changes.mady.by.user Shanthoosh Venkataraman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

After this change, we will have one method in TaskNameGrouper interface clearly defining the contract and all other methods in TaskNameGrouper will be deprecated(eventually removed). Host aware task to stream processors assignment in standalone will be housed in a TaskNameGrouper implementation which will be used to support this feature.
Implementation and Test Plan

  • Modify the existing interfaces and classes as per the proposed solution.

  • Add unit tests to test and validate compatibility and functional correctness.

  • Add a integration test in samza standalone samples to verify the host affinity feature.

  • Verify compatibility - Jackson, a java serialization/deserialization library is used to convert data model objects in samza into JSON and back. After removing containerId field from ContainerModel, it should be verified that deserialization of old ContainerModel data with new ContainerModel spec works. 

  • Some TaskNameGrouper implementations assumes the comparability of integer containerId present in ContainerModel(for instance - GroupByContainerCount, a TaskNameGrouper implementation). Modify existing TaskNameGrouper implementations to take in collection of string processorId’s, as opposed to assuming that containerId is integer and lies within [0, N-1] interval(without incurring any change in functionality).

Compatibility, Deprecation, and Migration Plan

  • In version 0.15, LocalityManager class will be changed into an interface. Any code which depends on LocalityManager class directly in open source should migrate to CoordinatorStreamBasedLocalityManager.
  • ContainerId field in ContainerModel which is deprecated in samza 0.13 version will be removed in the version 0.15. Open source users using containerId field from ContainerModel should migrate and use processorID field in ContainerModel.
  • All of the existing methods in TaskNameGrouper and BalancingTaskNameGrouper will be deprecated. 
  • It’s recommended that the users who migrate to samza version 0.15 should recompile their deployable due to the backward incompatible interface changes.

Rejected Alternatives

Approach 1

...