Motivation

...

• Is there any primitives, building blocks to provide compatibility?
• Difficulties for day by day coding.
  • java serialzation
    • anonymous class names based on declaration order
• Compatibility testing.
  • explicit tests.
  • ability to run tests with random node versions
• New feature implementation, enabling.
• patterns for implementing commons cases: new version of algorithmes, testing against new versions.
• Scope of compatibility: 
  • Currently any third-party module can register own ports (messages?) and must be able to track compatibility. 

Implementation phases

...

Code Cleanup

•  remove DirectByteBufferStreamImpl  V1-V3, keep V4, only.

...

•  remove all items from IgniteFeatures  and corresponding checks.

...

•  remove all code and checks for GridContinuousProcessor#discoProtoVer

...

•  TcpDiscoverySpi#setForceServerMode

...

•  All old version of classes - keep only max from V2, V3, etc. versions. StartRequestV2 that belongs to internal communication.
  Classes releated to thin client, jdbc, odbc interaction must stay.

...

• •  StartRequest must be deleted. Rename StartRequestV2 → StartRequest.

...

• •  CacheMetricsSnapshot must be deleted. Rename CacheMetricsSnapshotV2 → CacheMetricsSnapshot.

...

•  remove MessageFactory.

...

•  IgniteDataTransferObject
•  

  EXCHANGE_PROTOCOL_2_SINCE and related code

•  

  IgniteProductVersion.fromString - usages

Phase 1 - BinaryMarshaller modularization

...

BinaryMarshaller modularization

•  modularize BinaryMarshaller.
•  create small jar for ignite thin client

...

• . As a separate module, not part of the core. Optional, helps to see if changes affect resialization.
•  provide clear API for binary objects inside ignite-code and other modules. Finish IEP-119.

...

• Optional, helps to see if changes affect resialization.

...

Communication SPI Compatibility

•  Communication MessageWriter and MessageReader are aware of peers version, and then schema of a message.

...

Phase 3 - Discovery SPI Compatibility

•  Distinguish serdes generation from POJO

...

•  Guarantee that all messages use new serialization framework. Remove previous framework classes.
•  Code checks, see Communication protocol#Codechecks

Discovery SPI Compatibility

•   Serdes changes to provide compatibility? (current approach with JDK serialization provides some compatibility (13).
  Is it enough for long-term compatibility support?

...

•   Serdes changes to restrict classes that can be sent over SPI.
•  Guarantee that all messages use new serialization framework. Remove previous framework classes.
•  Code checks, see Communication protocol#Codechecks

Public API PR validation

• PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

Phase 4 - IgniteFeatures 

• Write down clear rules to deal with the new features and not compatible enhancements.
• Support, if not, already this rules in IgniteFeatures framework.

...

Management API

• Provide ability to change arg, result classes in compatible way.
  • Possible approach is to reuse communication serdes framework.
  • Other possibility is to migrate on BinaryObject as a arguments and results.
• PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

IgniteFeatures 

• Write down clear rules to deal with the new features and not compatible enhancements.

...

• Support, if not, already this rules in IgniteFeatures framework.

Compatibility Matrix

• A way to declare one version as incompatible with another.
• A component to enforce a rule that only compatible versions can join one cluster. 
• Only nodes with close versions (like 2.17 and 2.18, not 2.17 and 2.19) should be allowed to join cluster.

Testing

• ducktests to check upgrade procedure.
• unit tests mode that start some nodes of previous version. 

...

Development process changes

• Any change in public API MUST be in the form of IEP.
• Any change in public API MUST be voted by two(three?) committers.
• Documentation with clear description of development rules for all subsystems required to be compatible.
• Each release should be started with compatibility code removal.

Alternative designs

• subsystem API versions (like in REST API)
• runtime component(IgniteProcessor, IgniteManager) upgrade with the dynamic class loading. 

...