Page History

To be Reviewed By: August 19th 20th, 2020

Authors: Patrick Johnson

...

Geode is comprised of multiple sub-projects, many of which depend on other sub-projects at runtime to work correctly; these dependencies are defined in each project's build.gradle file. Normally, this is not a problem because all of Geode's projects are present on the Java classpath at runtime, allowing sub-projects to access classes from other sub-projects as required. However, the changes proposed by the ClassLoader Isolation RFC will result in all sub-projects being loaded as separate modules with different ClassLoaders and no longer being on the system classpath.

Once these modularizing changes go into effect, modules will no longer be able to access classes from other modules unless they explicitly create a dependency on them. In order to create dependencies between related modules, we need a way to determine at runtime which what modules depended depend on which what other modules, which, currently, we do not have. In addition to creating dependencies between modules, we may also need to load modules that are not already loaded , when loading a module that depends on them. For example, imagine the scenario where module A depends on modules B and C which both have dependencies of their own, represented by the below graphic.

...

Loading module A would require all of the modules in its dependency hierarchy (shown aboveB through F) to be loaded. If the modules required by module A are not already loaded, they would have to be loaded for module A to function correctly. Without knowledge of this hierarchy at runtime, we would be unable to ensure that all the modules required by module A are loaded when it is.

...

 The proposed solution is to add a manifest file with a "Dependent-Modules" attribute (or add a "Dependent-Modules" attribute to an existing manifest filefiles) to the jar files of all Geode sub-projects. The "Dependent-Modules" attribute will consist of a space-delimited list of sub-projects required at runtime by the current sub-project , in the form {name}-{version}. The "Dependent-Modules" attribute can then be read at runtime when loading the sub-project as a module and used to create the necessary dependencies on other modules. This list can be populated by a Gradle task at build time using the dependency information from each sub-project's build.gradle file, for example, these dependencies in a sub-project's build.gradle file:

compile(platform(project(':boms:geode-all-bom')))

api(project(':geode-core'))
api('org.apache.lucene:lucene-core')

compile(project(':geode-gfsh'))
implementation(project(':geode-logging'))
implementation(project(':geode-membership'))
implementation(project(':geode-serialization'))

implementation('org.apache.commons:commons-lang3')
implementation('org.apache.logging.log4j:log4j-api')

compileOnly(platform(project(':boms:geode-all-bom')))
compileOnly(project(':geode-common-services'))
compileOnly('com.fasterxml.jackson.core:jackson-annotations')

runtimeOnly('org.apache.lucene:lucene-analyzers-phonetic')

testImplementation(project(':geode-junit'))
testImplementation('org.apache.lucene:lucene-test-framework')

Would result in a "Dependent-Modules" attribute like this: 

Dependent-Modules: geode-gfsh-1.14.0-build.0 geode-core-1.14.0-build.0 geode-membership-1.14.0-build.0 geode-tcp-server-1.14.0-build.0 geode-http-service-1.14.0-build.0 geode-logging-1.14.0-build.0 geode-serialization-1.14.0-build.0 geode-common-1.14.0-build.0 geode-management-1.14.0-build.0 geode-unsafe-1.14.0-

...

build.0

Notice that only project dependencies and only those scoped to be required at runtime (runtime, runtimeOnly, compile, implementation, and api) are included in "Dependent-Modules". Other library dependencies are excluded because they will not be loaded as modules and dependencies scoped as compileOnly or test are excluded because they are not required to be accessed by modules at runtime. You may also notice that the "Dependent-Modules" list contains modules that are not explicitly included in Gradle, specifically geode-tcp-server, geode-http-service, geode-common, geode-management, and geode-unsafe. These modules are included in the list because they are required by other modules that are explicitly called out in Gradle and therefore, are indirectly required by the module. While this list accurately represents the modules that must be loaded, it is not hierarchical, which makes it more difficult to determine how this module fits into the overall picture of Geode" This list can be populated by a Gradle task at build time using the dependency information from each sub-project's build.gradle file. The "Dependent-Modules" attribute can then be read at runtime when loading the sub-project as a module and used to create the necessary dependencies on other modules.

The sample hierarchy shown in the graphic above is much cleaner than many of the actual relationships between sub-projects. Below shows a more realistic hierarchy structure.

, such as was shown by the sample Dependent-Modules attribute above. The below graph shows the actual relationships between all of Geode's sub-projects as defined by Gradle.

Image Added

As you can see, the above graph resembles a tangled ball of yarn, making it hard to determine any kind of meaningful hierarchy, or even what sub-projects are involved; a structure like this is difficult to read and unnecessarily complicated for creating dependencies between modules since a module can access other modules indirectly via its dependencies (e.g. if A depends on B and B depends on C, then A can access C via B ). As long as there is a Notice that the above hierarchy is not a tree, but a graph and that there are multiple paths between some modules, for example, A could get to C directly, or by going through D. While this may accurately represent the current dependencies between projects as they are defined in Gradle, it is unnecessarily complex given that modules can find each other via other modules; there only needs to be a single path between a module and the other modules it depends on, it can get what it needs, regardless of how many hops it takes. The above hierarchy dependency graph can be simplified at build-time by removing dependencies that are reachable via other dependencies from the dependency list of each sub-project. The remaining dependencies in the list become the value of the "Dependent-Modules" attribute in the sub-project's manifest file. By doing this, the above hierarchy graph can be simplified to something more like the diagram below. Gliffy Diagramsize600nameclean-hierarchypagePin1the following:

Image Added

This simplified dependency structure This simplified hierarchy has significantly fewer paths, making it easier to understand and to construct , while still allowing modules access to all of their dependencies. a module hierarchy from. Despite the difference in appearance between the above graph and the original, both allow the same access between modules.  Additionally, using this method of simplifying dependencies, the sample "Dependent-Modules" attribute given earlier would be reduced to this: 

Dependent-Modules: geode-gfsh-1.14.0-build.0

The reduced version of this attribute may be minimal, but it provides us all the information necessary to load and link the module.

Changes and Additions to Public Interfaces

...