...

This protocol is out of scope of this proposal.

Proposed changes

Cluster version

Validate joining node version

Server nodes stores ClusterVersion - set of Ignite versions in cluster (server and clients nodes). The joining node version must be checked versus the version (see actual Ignite versions in a cluster (within OnDiscoveryNodeValidationProcessor):

• If ClusterVersion a cluster contains nodes of only one version (all nodes are run same version), then the joining node can be greater/less than this version by up to 1 minor version.
• If ClusterVersion a cluster contains nodes of two versions, then joining node version must be one of the ClusterVersionsthem.

Message

...

serialization framework

There should be one serialization framework for communication and discovery protocol. 

1. MessageWriter, MessageReader logic is depends on a remote IgniteProductVersion.
2. Message#writeTo, Message#readFrom is auto-generated and stored separately from Message DTO classes classes.
3. Add Message#writeJavaObject Message#writeObject - for serializing Java Objects (java functions, and user objects (ComputeJob, etc) with JdkMarshaller)  with  BinaryMarshaller.
4. Message fields containfields may contain:
   1. primitive classes.
   2. known collections.
   3. POJO - that are other others Message.
   4. Users classes and java functions (e.g. ComputeJob).
   5. byte[] fields (objects serialized externally) must be avoided as much as possible, because their compatibility can't be guaranteed.
5. Order of fields in Message is fixed with @Order annotation.
6. Adding and removing fields must be go along with Add @Since, @Until  annotations  annotations for Message classes and fields.

...

Code checks

1. Ignite CI must notify notifies for IF-clauses with condition based on IgniteVersion older than (curVer - 1).
2. Ignite CI must forbid forbids code changes if Message DTO fields changed without corresponding @Since, @Until annotations.
3. Ignite CI must forbid forbids code changes if Message DTO contains byte[] fields.
4. Ignite CI forbids code changes if Message field changes type.
5. Ignite CI checks @Order annotation of fields - starts with 0, no lags.

Nice to have (for later research)

...

1. Data - set of declared Message classes, including ser/des algorithm.
2. Transport - algorithm of transport the Messages to remote node.

Data compatibility

Message - is base class for all messages transported between nodes. Proposed changes:

1. Deprecate Message#writeTo and Message#readFrom in favor generated MessageSerializer. 
2. MessageSerializer#writeTo Message#writeTo consumes MessageWriter that stores IgniteProduceVersion IgniteProductVersion of destination a receiver node and use it for serializing data for this version (mostly, for ignoring some fields).

3. Message#readFrom MessageSerializer#readFrom consumes MessageReader that stores IgniteProductVersion of source node and use it for deserializing data (mostly, for setting default values of new fields).

   Code Block
   language java title Message
   public interface MessageMessageSerializer {    public boolean writeTo(Message msg, ByteBuffer buf, MessageWriter writer);     public boolean readFrom(Message msg, ByteBuffer buf, MessageReader reader);     } public interface MessageWriter { public IgniteProductVersion receiverVersion(); } public interface MessageReader { public shortIgniteProductVersion directTypesenderVersion(); }

   
   

4. Introduce annotations @Since and @Until for Message classes and Message fields, to use it for generating code for Message#writeTo and Message#readFrom:
   
   

// Package thatwhere the storeMessage allis schemasdefined.
package org.apache.ignite.internal.messagesmy.schemamessage;

// package private class.
@Since(version = "2.19.0")
public class MyMessage implements Message {
    @Order(0)
    private int id;
    
 	/** Remove field. */
    @Until(version = "2.20.0")
    @Order(1)
    private String rmFld;

    /** New field. */
    @Since(version = "2.20.0")
 	@Order(2)
    private String newFld;
}

	// Message Generated code from the schema ^. 
package org.apache.ignite.internal.messages;

// public class.
public class MyMessage {
    private int id;
    
	/**
     * Remove field. 
     * @deprecated since 2.20.0. 
     */
    @Deprecated
    private String rmFld;

    /** 
     * New field.
     * @since 2.20.0
     */
    private String newFld;

    @Override public boolean writeTo(must have setters/getters for all @Ordered fields. Methods names are equal to a corresponding field name. 
	public void id(int id) {
		this.id = id;
	}

  	public int id() {
		return id;
	}

  	public void newFld(String newFld) {
		this.newFld = newFld;
	}
 
 	public String newFld() {
		return newFld;
	}
}

// Generated code from the message ^ for Ignite version 2.20.0.
// Use the same package as corresponding message.
package org.apache.ignite.internal.my.message;

class MyMessageSerializer {
    public boolean writeTo(Message m, ByteBuffer buf, MessageWriter writer, IgniteProductVersion destVer) {
		MyMessage        if (destVer.lessThan(2, 19, 0))
            throw new IgniteException("Must not send the message to destination node"msg = (MyMessage)m;

		IgniteProductVersion rcvVer = writer.receiverVersion();

        if (!writer.writeString(msg.id())
            return false;

        if (destVerrcvVer.lessThan(2, 20, 0)) {
            if (!writer.writeString(msg.rmFld())
                return false;
        }

        if (destVerrcvVer.greaterThanEqual(2, 20, 0)) {
            if (!writer.writeString(msg.newFld())
                return false;
        }

        return true;
    }

    @Overridepublic publicstatic boolean readFrom(Message m, ByteBuffer buf, MessageReader reader, IgniteProductVersion srcVer) {
) {
 		MyMessage msg = (MyMessage)m;
 
        IgniteProductVersion idsrcVer = reader.readStringsenderVersion();

        msg.id(reader.readString());

		if (srcVer.greaterThanEquallessThan(2, 20, 0))
            newFld = 			msg.rmFld(reader.readString());

        if (srcVer.greaterThanEqual(2, 20, else0))
            newFld = nullmsg.newFld(reader.readString());

        return true;
    }
}

Rules to describe Message (must be automated and validated):

1. Do not remove Message class or Message fields, but annotate it with @Until. 
2. Do not change types or order @Order of fields.
3. New fields must be annotated with @Since. All such fields must be optional, default value is null. Handling the nulls is care of Message consumer on the reader side.

...

1. Setters and getters must follow name of the field.

Transport compatibility 

Communication handshake

...