Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Is there any primitives, building blocks to provide compatibility?
  • Difficulties for day by day coding.
  • Compatibility testing.
    • explicit tests.
    • ability to run tests with random node versions
  • New feature implementation, enabling.
  • patterns for implementing commons cases: new version of algorithmes, testing against new versions.
  • Scope of compatibility: 
    • Currently any third-party module can register own ports (messages?) and must be able to track compatibility. 

...

Implementation phases

Phase 0 - Code Cleanup

  • remove DirectByteBufferStreamImpl  V1-V3, keep V4, only.
  • remove all items from IgniteFeatures  and corresponding checks.
  • remove all code and checks for GridContinuousProcessor#discoProtoVer
  • TcpDiscoverySpi#setForceServerMode
  • All old version of classes - keep only max from V2, V3, etc. versions. StartRequestV2 that belongs to internal communication.
    Classes releated to thin client, jdbc, odbc interaction must stay.
    • StartRequest must be deleted. Rename StartRequestV2 → StartRequest.
    • CacheMetricsSnapshot must be deleted. Rename CacheMetricsSnapshotV2 → CacheMetricsSnapshot.
  • MessageFactory.

Phase 1 - BinaryMarshaller modularization

  • modularize BinaryMarshaller.
  • create small jar for ignite thin client
  • provide clear API for binary objects inside ignite-code and other modules.

Phase 2 - Communication SPI Compatibility

  • Use fieldNumber  instead of fieldName provide compatible messages.
  • (question) Ability to enable compatible Communication in runtime because of fieldNumber  overhead.
  • (question) Distinguish serdes generation from POJO
  • PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

Phase 3 - Discovery SPI Compatibility

  • (question) Serdes changes to provide compatibility? (current approach with JDK serialization provides some compatibility (13).
    Is it enough for long-term compatibility support?
  • (question) Serdes changes to restrict classes that can be sent over SPI.
  • PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

Phase 4 - IgniteFeatures 

  • Write down clear rules to deal with the new features and not compatible enhancements.
  • Support, if not, already this rules in IgniteFeatures framework.

Phase 5 - Management API

  • Provide ability to change arg, result classes in compatible way.
    • Possible approach is to reuse communication serdes framework.
    • Other possibility is to migrate on BinaryObject as a arguments and results.
  • PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

Alternative designs

  • subsystem API versions (like in REST API)
  • runtime component(IgniteProcessor, IgniteManager) upgrade with the dynamic class loading. 

...

  1. https://www.datastax.com/learn/whats-new-for-cassandra-4/migrating-cassandra-4x
  2. https://docs.datastax.com/en/luna-cassandra/guides/upgrade/overview.html
  3. https://github.com/apache/cassandra-dtest/blob/trunk/upgrade_tests/README.md
  4. https://kafka.apache.org/documentation/#upgrade
  5. https://github.com/apache/kafka/blob/trunk/clients/src/main/resources/common/message/AlterPartitionResponse.json
  6. https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/core/kraft_upgrade_test.py
  7. https://docs.yugabyte.com/preview/manage/upgrade-deployment/
  8. https://github.com/yugabyte/yugabyte-db/commit/e9ab17dea0d3b4f1673531c07404a290f9fbd8f2
  9. https://github.com/ydb-platform/ydb/blob/main/ydb/tests/functional/restarts/
  10. IEP-119 Binary infrastructure modularization
  11. PDS Compatibility Guide (WIP)
  12. https://hazelcast.com/products/rolling-upgrade/
  13. https://docs.oracle.com/en/java/javase/17/docs/specs/serialization/version.html#compatible-java-type-evolution