To be Reviewed By: August 20th, 2020
Authors: Patrick Johnson
Status: Draft | Discussion | Active | Dropped | Superseded
Superseded by: N/A
Related: ClassLoader Isolation
Problem
Geode is comprised of multiple sub-projects, many of which depend on other sub-projects at runtime; these dependencies are defined in each project's build.gradle file. Normally, this is not a problem because all of Geode's projects are present on the Java classpath at runtime, allowing sub-projects to access classes from other sub-projects as required. However, the changes proposed by the ClassLoader Isolation RFC will result in all sub-projects being loaded as separate modules with different ClassLoaders and no longer being on the system classpath.
Once these modularizing changes go into effect, modules will no longer be able to access classes from other modules unless they explicitly create a dependency on them. In order to create dependencies between related modules, we need a way to determine at runtime what modules depend on what other modules, which, currently, we do not have. In addition to creating dependencies between modules, we may also need to load modules that are not already loaded when loading a module that depends on them. For example, imagine the scenario where module A depends on modules B and C which both have dependencies of their own, represented by the below graphic.
Loading module A would require all of the modules in its dependency hierarchy (B through F) to be loaded. If the modules required by module A are not already loaded, they would have to be loaded for module A to function correctly. Without knowledge of this hierarchy at runtime, we would be unable to ensure that all the modules required by module A are loaded when it is.
Anti-Goals
This proposal is intended only to solve the above problem of determining at runtime, which sub-projects/modules depend on which other sub-projects/modules and is not concerned with...
- The implementation of modules in Geode
- Refactoring Geode's sub-projects to break or change any existing dependencies between them
- Adding or removing sub-projects
Solution
The proposed solution is to add a manifest file with a "Dependent-Modules" attribute (or add a "Dependent-Modules" attribute to existing manifest files) to the jar files of all Geode sub-projects. The "Dependent-Modules" attribute will consist of a space-delimited list of sub-projects required at runtime by the current sub-project in the form {name}-{version}. The "Dependent-Modules" attribute can then be read at runtime when loading the sub-project as a module and used to create the necessary dependencies on other modules. This list can be populated by a Gradle task at build time using the dependency information from each sub-project's build.gradle file, for example, these dependencies in a sub-project's build.gradle file:
compile(platform(project(':boms:geode-all-bom')))
api(project(':geode-core'))
api('org.apache.lucene:lucene-core')
compile(project(':geode-gfsh'))
implementation(project(':geode-logging'))
implementation(project(':geode-membership'))
implementation(project(':geode-serialization'))
implementation('org.apache.commons:commons-lang3')
implementation('org.apache.logging.log4j:log4j-api')
compileOnly(platform(project(':boms:geode-all-bom')))
compileOnly(project(':geode-common-services'))
compileOnly('com.fasterxml.jackson.core:jackson-annotations')
runtimeOnly('org.apache.lucene:lucene-analyzers-phonetic')
testImplementation(project(':geode-junit'))
testImplementation('org.apache.lucene:lucene-test-framework')
Would result in a "Dependent-Modules" attribute like this:
Dependent-Modules: geode-gfsh-1.14.0-build.0 geode-core-1.14.0-build.0 geode-membership-1.14.0-build.0 geode-tcp-server-1.14.0-build.0 geode-http-service-1.14.0-build.0 geode-logging-1.14.0-build.0 geode-serialization-1.14.0-build.0 geode-common-1.14.0-build.0 geode-management-1.14.0-build.0 geode-unsafe-1.14.0-build.0
Notice that only project dependencies and only those scoped to be required at runtime (runtime, runtimeOnly, compile, implementation, and api) are included in "Dependent-Modules". Other library dependencies are excluded because they will not be loaded as modules and dependencies scoped as compileOnly or test are excluded because they are not required to be accessed by modules at runtime. You may also notice that the "Dependent-Modules" list contains modules that are not explicitly included in Gradle, specifically geode-tcp-server, geode-http-service, geode-common, geode-management, and geode-unsafe. These modules are included in the list because they are required by other modules that are explicitly called out in Gradle and therefore, are indirectly required by the module. While this list accurately represents the modules that must be loaded, it is not hierarchical, which makes it more difficult to determine how this module fits into the overall picture of Geode.
The sample hierarchy shown in the graphic above is much cleaner than many of the actual relationships between sub-projects, such as was shown by the sample Dependent-Modules attribute above. The below graph shows the actual relationships between all of Geode's sub-projects as defined by Gradle.
As you can see, the above graph resembles a tangled ball of yarn, making it hard to determine any kind of meaningful hierarchy, or even what sub-projects are involved; a structure like this is difficult to read and unnecessarily complicated for creating dependencies between modules since a module can access other modules indirectly via its dependencies (e.g. if A depends on B and B depends on C, then A can access C via B ). As long as there is a path between a module and the other modules it depends on, it can get what it needs, regardless of how many hops it takes. The above dependency graph can be simplified at build-time by removing dependencies that are reachable via other dependencies from the dependency list of each sub-project. The remaining dependencies in the list become the value of the "Dependent-Modules" attribute in the sub-project's manifest file. By doing this, the above graph can be simplified to the following:
This simplified dependency structure has significantly fewer paths, making it easier to understand and to construct a module hierarchy from. Despite the difference in appearance between the above graph and the original, both allow the same access between modules. Additionally, using this method of simplifying dependencies, the sample "Dependent-Modules" attribute given earlier would be reduced to this:
Dependent-Modules: geode-gfsh-1.14.0-build.0
The reduced version of this attribute may be minimal, but it provides us all the information necessary to load and link the module.
Changes and Additions to Public Interfaces
No anticipated changes to public interfaces.
Performance Impact
No anticipated performance impact.
Backwards Compatibility and Upgrade Path
No backward compatibility impact.
Prior Art
An alternative to this proposal would be to load every sub-project as a module at startup and create dependencies between every module and every other module. While this may solve the problem, it would also largely defeat the purpose of modularizing Geode in the first place.
FAQ
Answers to questions you’ve commonly been asked after requesting comments for this proposal.
Errata
What are minor adjustments that had to be made to the proposal since it was approved?