...

•  remove DirectByteBufferStreamImpl  V1-V3, keep V4, only.
•  remove all items from IgniteFeatures  and corresponding checks.
•  remove all code and checks for GridContinuousProcessor#discoProtoVer
•  TcpDiscoverySpi#setForceServerMode
•  All old version of classes - keep only max from V2, V3, etc. versions. StartRequestV2 that belongs to internal communication.
  Classes releated to thin client, jdbc, odbc interaction must stay.
  •  StartRequest must be deleted. Rename StartRequestV2 → StartRequest.
  •  CacheMetricsSnapshot must be deleted. Rename CacheMetricsSnapshotV2 → CacheMetricsSnapshot.
•  remove MessageFactory.
•  IgniteDataTransferObject
•  

  EXCHANGE_PROTOCOL_2_SINCE and related code

•  

  IgniteProductVersion.fromString - usages

...

•  modularize BinaryMarshaller.
•  create small jar for ignite thin client. As a separate module, not part of the core. Optional, helps to see if changes affect resialization.
•  provide clear API for binary objects inside ignite-code and other modules. Finish IEP-119. Optional, helps to see if changes affect resialization.

...

•  Communication MessageWriter and MessageReader are aware of peers version, and then schema of a message.
•  Distinguish serdes generation from POJO
•  Guarantee that all messages use new serialization framework. Remove previous framework classes.
•  Code checks, see Communication protocol#Codechecks

Discovery SPI Compatibility

•   Serdes changes to provide compatibility? (current approach with JDK serialization provides some compatibility (13).
  Is it enough for long-term compatibility support?
•   Serdes changes to restrict classes that can be sent over SPI.
•  PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.Guarantee that all messages use new serialization framework. Remove previous framework classes.
•  Code checks, see Communication protocol#Codechecks

...

• Provide ability to change arg, result classes in compatible way.
  • Possible approach is to reuse communication serdes framework.
  • Other possibility is to migrate on BinaryObject as a arguments and results.
• PR checks for Messages  ancestor change with the some warning, labels, etc. that can draw reviewer attention to the possible compatibility issues.

IgniteFeatures 

• Write down clear rules to deal with the new features and not compatible enhancements.
• Support, if not, already this rules in IgniteFeatures framework.

Compatibility Matrix

• A way to declare one version as incompatible with another.
• A component to enforce a rule that only compatible versions can join one cluster. 
• Only nodes with close versions (like 2.17 and 2.18, not 2.17 and 2.19) should be allowed to join cluster.

Testing

• ducktests to check upgrade procedure.
• unit tests mode that start some nodes of previous version. 

...

• Any change in public API MUST be in the form of IEP.
• Any change in public API MUST be voted by two(three?) committers.
• Documentation with clear description of development rules for all subsystems required to be compatible.
• Each release should be started with compatibility code removal.

Alternative designs

• subsystem API versions (like in REST API)
• runtime component(IgniteProcessor, IgniteManager) upgrade with the dynamic class loading. 

...