Move membership code to a separate gradle sub-project
To be Reviewed By: 9/6/2019
Authors: Dan Smith, Bruce Schuchardt
Status: Discussion Development
Superseded by: N/A
Related: N/A
...
PlantUML |
---|
@startuml "geode-core" --> geode-membership "geode-core" --> geode-serialization "geode-core" --> geode-tcp-server "geode-membership" --> geode-serialization "geode-membership" --> geode-tcp-server "geode-tcp-server" --> geode-serialization @enduml |
Breaking circular dependencies
...
- Move the dependencies to a separate subproject - serialization is and tcp-server (Locator infrastructure) are moving to separate subproject subprojects so that membership can depend directly on thatthem
- Inject dependencies at runtime - dependencies such as gemfire stats can be injected into the membership system by providing interfaces such as MembershipStatistics (see below) that geode core must provide when creating the membership system.
...
PlantUML |
---|
@startuml interface MembershipBuilder { + static newMembershipBuilder() ---- + set...() } interface Membership { Core class of a running membership system. + getMembershipView(): MembershipView } interface Messenger { + send(Message) } interface MembershipView { + getMembers(): List<MemberID>List<MemberIdentifier> } interface MemberIDMemberIdentifier { A single member of the system } interface MembershipListener { } interface MessageHandler { } interface Authenticator { } interface MembershipStatistics { } interface Config { } interface MembershipView { } MembershipBuilder --> Membership : creates MembershipBuilder *-- Authenticator MembershipBuilder *-- MembershipStatistics MembershipBuilder *-- MessageHandler MembershipBuilder *-- Config MembershipBuilder *-- MembershipListener Membership *-- Messenger Membership *-- MembershipView MembershipView *-- MemberIDMemberIdentifier @enduml |
Code Block | ||
---|---|---|
| ||
/** * Creates the membership system, given the provided configuration */ interface MembershipBuilder { static MembershipBuilder newMembershipBuilder() MembershipBuilder setConfig(Config) MembershipBuilder setAuthenticator(Authenticator) MembershipBuilder setMembershipListener(MembershipListener) MembershipBuilder setMessageHandler(MessageHandler) MembershipBuilder setMembershipStatistics(MembershipStatistics) setMemberFactoryMembershipBuilder setMemberIDFactory(MemberIDFactoryMemberIdentifierFactory) create() : Membership } /** * Core class of a running membership system. */ interface Membership { getMembershipView(): MembershipView getMessenger(): Messenger getLocalMember(): MemberIDMemberIdentifier contactedBy(MemberIDMemberIdentifier) isShunned(MemberIDMemberIdentifier) close() isClosed() } /** * API for sending messages to other members, using memberships messaging system * Membership currently allows geode-core to send messages over it's UDP messaging system. * This interface provides that functionality. */ interface Messenger { send(Message) /** Get an object the represents thewhat messages that have been sent to to the given * through this* messengerMemberIdentifier. This is used in waitForMessageState * TowaitForMessengerState to wait for these messages * to arrive on the remote side side */ getMessageStategetMessengerState(MemberIdentifer): ObjectMessengerState waitForMessageStatewaitForMessengerState(ObjectMessengerState) } /** * InterfaceA usedMessage tothat createwill abe MemberIDsent fromto athe setreturned ofrecipients. properties */ interface ThisMessage API{ is provided solely to allow geode-core to add additional * properties to the MemberID that are not relevant to membership * For example DurableClientAttributes. Membership will provide * A default MemberID which just has the membership relevant attributes */ interface MemberIDFactory { MemberID create(host, port, uuid, ...) } /** * A single member of the system. In practice, this will * be implemented by InternalDistributedMember */ interface MemberID { } /** * Provides the current members of the system */ interface MembershipView { List<MemberID> getMembers() } /** * Receives notifications about changes to membership */ interface MembershipListener { memberJoined(MemberID) memberDeparted(MemberID) forceDisconnected() } /** * Receives all messages sent from other members */ interface MessageHandler { processMessage(MessageList<MemberIdentifier> getRecipients() } /** * An object which represents the state of the sending side of a communication * channel at a point in time. This object can be passed to the waitForMessengerState * method on the receiving side to ensure all messages have been received on * the receiving side */ interface MessengerState extends DataSerializableFixedID { }/** * Interface used to create a MemberIdentifier from a set of properties * This API is provided solely to allow geode-core to add additional * properties to the MemberIdentifier that are not relevant to membership * For example DurableClientAttributes. Membership will provide * A default MemberIdentifier which just has the membership relevant attributes */ interface MemberIdentifierFactory { MemberIdentifier create(memberData) } /** * InterfaceA usedsingle bymember membership to authenticate other membersof the system. In practice, this will * be implemented by InternalDistributedMember */ interface AuthenticatorMemberIdentifier { } /** * InterfaceProvides tothe notifycurrent statisticsmembers systemsof aboutthe membership changessystem */ interface MembershipStatisticsMembershipView { List<MemberIdentifier> memberJoinedgetMembers() memberDeparted() } /** * PrimitiveReceives configurationnotifications optionsabout forchanges to membership * - timeouts, etc. */ interface Config { getJoinTimeout() .../ interface MembershipListener { memberJoined(MemberIdentifier) memberDeparted(MemberIdentifier) memberCrashed(MemberIdentifier) forceDisconnected(String reason) } /** |
geode-serialization API
The geode-serialization subproject will provide the following API to the rest of the system. We will enforce that other parts of the system can only interact with this API.
Internally Geode uses DataSerializableFixedID as the interface for most messages and data objects. The membership system uses this exclusively. We will separate the serialization framework for this interface and leave the rest (PDX, DataSerializable, etc.) in geode-core.
DataSerializableFixedID currently has the same signature for its toData and fromData methods but this will be changed to take another parameter, a serialization context. The context provides access to serializers and information about the source/destination (currently the peer's version).
* Receives all messages sent from other members
*/
interface MessageHandler {
processMessage(Message)
}
/**
* Interface used by membership to authenticate other members
*/
interface Authenticator {
/**
* Authenticate peer member
*
* @return null if authentication succeed (including no authenticator case), otherwise, return
* failure message
*/
String authenticate(MemberIdentifier member, Properties credentials);
/**
* Get credential object for the given GemFire distributed member.
*/
Properties getCredentials(MemberIdentifier member);
}
/**
* Interface to notify statistics systems about membership changes
*/
interface MembershipStatistics {
memberJoined()
memberDeparted()
...
}
/**
* Primitive configuration options for membership
* - timeouts, etc.
*/
interface Config {
getJoinTimeout()
...
}
|
geode-serialization API
The geode-serialization subproject will provide the following API to the rest of the system. We will enforce that other parts of the system can only interact with this API.
Internally Geode uses DataSerializableFixedID as the interface for most messages and data objects. The membership system uses this exclusively. We will separate the serialization framework for this interface and leave the rest (PDX, DataSerializable, etc.) in geode-core.
DataSerializableFixedID currently has the same signature for its toData and fromData methods but this will be changed to take another parameter, a serialization context. The context provides access to serializers and information about the source/destination (currently the peer's version).
Code Block | ||
---|---|---|
| ||
/** Use DSFIDSerializerFactory to create a serializer */
class DSFIDSerializerFactory {
/** set an ObjectSerializer that should be used when invoking
* toData/fromData methods on objects. This will typically
* defer serialization of DataSerializableFixedID objects to
* the DSFIDSerializer
*/
DSFIDSerializerFactory setObjectSerializer(ObjectSerializer serializer);
DSFIDSerializerFactory setObjectDeserializer(ObjectDeserializer deserializer);
DSFIDSerializer create();
}
/** writes and reads DataSerializableFixedID instances. Supports reading/writing of nulls */
interface DSFIDSerializer {
/** retrieve the ObjectSerializer. If none was given to the factory this will be
* a default serializer that can only handle DataSerializableFixedID objects */
ObjectSerializer getObjectSerializer();
ObjectDeserializer getObjectDeserializer();
/** register a class with the serializer. It must have a no-arg constructor */
void registerDSFID(int dsfid, Class dsfidClass);
SerializationContext createSerializationContext(DataOutput dataOutput);
DeserializationContext createDeserializationContext(DataInput dataInput);
/** write a DataSerializableFixedID to the given DataOutput. This method should be used
* by extensions to DSFIDSerializer, like InternalDataSerializer, that want to handle
* more types in ObjectSerializer but defer DSFID serialization to the
* default implementation */
void writeDSFID(DataSerializableFixedID o, int dsfid, DataOutput out) throws IOException | ||
Code Block | ||
| ||
/** Use DSFIDSerializerFactory to create a serializer */ class DSFIDSerializerFactory { /** set an ObjectSerializer that should be used when invoking * toData/fromData methods on objects. This will typically * defer serialization of DataSerializableFixedID objects to * the DSFIDSerializer */ DSFIDSerializerFactory setObjectSerializer(ObjectSerializer serializer); DSFIDSerializer create(); } /** writes and reads DataSerializableFixedID instances. Supports reading/writing of nulls */ interface DSFIDSerializer { /** retrieve the ObjectSerializer. If none was given to the factory this will be * a default serializer that can only handle DataSerializableFixedID objects */ ObjectSerializer getObjectSerializer(); /** registerread a classDataSerializableFixedID withfrom the given serializerDataInput. ItThis mustmethod haveuses aconstructors no-arg constructor */ registered voidvia registerDSFID(int dsfid, Class dsfidClass); SerializationContext createSerializationContext(DataOutput dataOutput); DeserializationContext createDeserializationContext(DataInput dataInput); void writeDSFID(DataSerializableFixedID o, int dsfid, DataOutput out) throws IOException; void writeDSFID(DataSerializableFixedID dsfid, DataOutput out) throws IOException; ) and should be used by extensions to DSFIDSerializer, * like InternalDataSerializer, that want to handle more types in ObjectDeserializer but * defer DSFID serialization to the default implementation */ Object create(int dsfid, DataInput in) throws IOException, ClassNotFoundException; void invokeToData(Object ds, DataOutput out) throws IOException; void invokeFromData(Object ds, DataInput in) throws IOException, ClassNotFoundException; int readDSFIDHeader(DataInput dis) throws IOException; void writeDSFIDHeader(int dsfid, DataOutput out) throws IOException; } /** provides context for toData serialization methods */ interface SerializationContext { ObjectSerializer getSerializer(); Version getSerializationVersion(); } /** provides context for toData serialization methods */ interface DeserializationContext { ObjectSerializerObjectDeserializer getSerializergetDeserializer(); Version getSerializationVersion(); } interface ObjectSerializer { public void writeObject(Object obj, DataOutput output) throws IOException; void writeObjectinvokeToData(Object objds, DataOutput outputout) throws IOException; ... } interface ObjectDeserializer { public Object readObject(DataInput input) throws IOException, ClassNotFoundException; void invokeToData(Object ds, DataOutput out) throws IOException; void invokeFromData(Object ds, DataInput in) throws IOException, ClassNotFoundException; public Version getVersionForOrdinalOrCurrent(int ordinal); ... } /** a new superclass of HeapDataOutputStream that separates the methods used with Buffers from a bunch of other methods concerning other functionality such as Streamables, PDX, etc. */ class BufferDataOutputStream implements DataOutputStream { // methods moved from HeapDataOutputStream unchanged } /** DataSerializableFixedID is modified to have more parameters in toData/fromData methods */ interface DataSerializableFixedID { public Version[] getSerializationVersions(); public int getDSFID(); public void toData(DataOutput output, SerializationContext context) throws IOException; public void fromData(DataInput input, DeserializationContext context) throws ClassNotFoundException, IOException; } |
...
Use DSFIDSerializerFactory to create a new serializer. Set an ObjectSerializer/ObjectDeserializer if you want to provide serialization of other types of objects in readObject/writeObject. Use DSFIDSerializer.registerClass() to register your classes so that the serializer knows how to instantiate them.
The new DSFID serializer sub-project will by and large not be composed of static methods as is InternalDataSerializer and its superclass DataSerializer. InternalDataSerializer will hold an instance of the DSFIDSerializer and will register DSFID codes and constructors with it. Instantiation will insert an ObjectSerializer into ObjectSerializer and ObjectDeserializer into the DSFIDSerializer to provide users of the serializer the ability to write PDX, DataSerializables and other interfaces serialization types supported by Geode but not (directly) by the DSFIDSerializer. InternalDataSerializer and DataSerializer will defer serialization of DSFIDs to the DSFIDSerializer instance.DSFIDs to the DSFIDSerializer instance. The geode-core DSFIDFactory will register all geode-core DataSerializableFixedID classes with the instance of the serializer held by InternalDataSerializer, as will other Geode subprojects like geode-wan.
While serialization of primitives and Strings can be performed using DataInput and DataOutput there are a A number of static serialization methods in DataSerializer and InternalDataSerializer will also currently in geode-core that will have to be shifted into the DSFIDSerializer (writeString/readString, writeHashmap/readHashmap, etc) to maintain compatibility with past releases. Those methods shifted from DataSerializer will remain available in that class since it is a public API. The intent is to shift these methods further into the ObjectSerializer/ObjectDeserializer APIs and stop using static serialization methods.
Since Version is being repackaged any rolling-upgrade test code that refers to it will no longer work when running with an older version of Geode. We will provide methods in the distributed test framework VersionManager to make this less painful. We will replace the use of Version.CURRENT_ORDINAL with VersionManager.getInstance().getCurrentVersionOrdinal() in these tests. Other methods will be added to VersionManager as needed.
Changes and Additions to Public Interfaces
...