Move membership code to a separate gradle sub-project

To be Reviewed By:

Authors:

Status: Draft | Discussion | Active | Dropped | Superseded

Superseded by: N/A

Related: N/A

Problem

Geode has its own custom membership system, which is responsible for discovering other members of the system, and for detecting and removing failed members. We want to be able to exhaustively test that the membership system is working correctly in all circumstances, However, this membership system is not cleanly separated from the rest of the geode code. This makes it difficult and expensive to test, because we have to create full geode members and connect them over a network in order to test any interaction between the membership system of two geode members.

By isolating the membership code such that it does not depend on the rest of the geode, we can test the membership system by itself. This will allow us to write much faster and more exhaustive tests of how multiple members interact.

Creating a well defined internal API for the membership system and hiding the internals from the rest of geode will also make it easier for geode developers to reason about what the membership system is and is not doing while writing and testing other components of geode.

Anti-Goals

• It is not a goal to give the membership system a public API for use by geode users. The membership system will have an internal  API and be usable in isolation, but we don't intend to advertise it as a publicly available component by itself.
• It is not a goal to create an SPI that allows different membership systems to be swapped in.

Solution

Describe your solution and how it’s going to solve the problem. This is likely the largest section of your proposal and might even include some high-level diagrams if you are proposing code changes. While all important aspects need to be covered, also keep in mind that shorter documents are more likely to be read.

We will create a new gradle subproject called geode-membership. All of the code in the org.apache.geode.distributed.internal.membership.gms and related unit tests will be moved out of geode-core and into geode-membership. geode-core will have a dependency on geode-membership, and interact with membership through its API, defined below.

...

@startuml

...

...

class

...

Classes used to wrap the membership APIs or inject dependencies into membership will be put in the org.apache.geode.distributed.internal.membership.adapter package.

TODO

...

The geode-membership subproject will provide the following API to the rest of the system. We will enforce that other parts of the system can only interact with this API.

...

 MembershipManagerFactory {

...

  Creates the membership system, given the provided configuration

...

  + setConfig(GMSConfig)

...

  + setAuthenticator(GMSAuthenticator)

...

  + setMembershipListener(GMSMembershipListener)

...

  + setMessageHandler(GMSMessageHandler)

...

  + setStatsListener(GMSStatsListener)

...

  + create() : MembershipManager

...

}

...

interface MembershipManager {

...

Core class of a running membership system.

...

  + getMembershipView(): GMSMembershipView

...

  + getMessenger(): Messenger

...

  + getLocalMember(): GMSMember

...

  + close()

...

  + isClosed()

...

}

...

interface Messenger {

...

  + send(GMSMessage)

...

  + getMessageState(): Object

...

  + waitForMessageState(Object)

...

}

...

interface GMSMembershipView {

...

  Provides the current members of the system

...

}

...

interface GMSMember {

...

  A single member of the system

...

}

...

interface GMSMembershipListener {

...

  Receives notifications about changes to membership

...

  + memberJoined(GMSMember)

...

  + memberDeparted(GMSMember)

...

  + forceDisconnected()
}
interface GMSMessageHandler {

...

  Receives all messages sent from other members

...

  + processMessage(GMSMessage)

...

}

...

interface GMSAuthenticator {

...

  Interface used by membership to authenticate other members

...

}

...

interface GMSStatsListener {

...

  Interface to notify statistics systems about membership changes

...

  +memberJoined()

...

  +memberDeparted()

...

}

...

interface GMSConfig {

...

  Configuration options for membership

...

  + setJoinTimeout()

...

  + {method} ...

...

}

...

MembershipManagerFactory --> MembershipManager : creates

...

MembershipManagerFactory *-- GMSMessageHandler

...

MembershipManagerFactory *-- GMSMembershipListener

...

MembershipManagerFactory *-- GMSAuthenticator

...

MembershipManagerFactory *-- GMSStatsListener

...

MembershipManagerFactory *-- GMSConfig

...

MembershipManager *-- Messenger

...

MembershipManager *-- GMSMembershipView

...

MembershipManager *-- GMSMember

...

TODO

geode-serialization API

The geode-serialization subproject will provide the following API to the rest of the system. We will enforce that other parts of the system can only interact with this API.

TODO

Changes and Additions to Public Interfaces

This proposal does not add or remove any public API.

Performance Impact

The intention of this proposal is to not change the performance of geode significantly.

Backwards Compatibility and Upgrade Path

...

@enduml

Prior Art

...

As an alternative, we could isolate the membership code without creating a new gradle subproject and geode-membership jar file. However without enforcing what the membership code can and cannot depend on, it is likely that the membership code will not stay isolated as different developers work on the code.

FAQ

Q: Why not introduce a separate messenging module, and leave things like Messenger out of geode-membership?

A: We may eventually try to split messenging out of membership, but that is not part of the current proposal. The reason membership includes messaging in this proposal is that the product currently uses the messaging system bootstrapped by membership to send arbitrary messages. Membership requires a lower level messaging system (jgroups) but it uses that to provide a higher level messaging system to the rest of the system. The higher level messaging provided by membership includes additional features such as cluster wide encryption and ensuring that we only process messages from current members.
Answers to questions you’ve commonly been asked after requesting comments for this proposal.

Errata

...