Geode is a toolkit for Java—it is not a server product.

Status

Legacy

Context

Shall Geode be a server product or a (Java) toolkit?

Traditional client-server database and web application caching products provide thin or fat client libraries. In those systems there is a clear separation between clients, which run user-defined code plus a client library/framework, and servers, which run in processes defined wholly by product code. If those servers provide for execution of user-defined code, that execution happens solely under the control of product code.

Users of these traditional client-server products (referred to as "server products" here) are shielded from server-side implementation choices such as product implementation language, so long as suitable client libraries are available. This is a key choke-point in such architectures. By narrowing client-server communication through a (assumed small, stable) network protocol—many different client technology stacks can be supported, and much freedom to evolve the server implementation is gained.

The Geode product documentation describe two topologies: client-server, and peer-to-peer.  The client-server topology aligns most closely with what administrators and developers coming from widely-used database products and web application caching products have come to expect. But the so-called peer-to-peer support in Geode offers a wide variety of configuration possibilities beyond that.

As a low-latency, consistent in-memory caching product, Geode must (almost by definition) provide at least a "fat client" library. In order to provide low-latency, consistent caching, changes must be propagated synchronously, and data must be made available in-process (in the user application process). Clients cannot, in general, wait for a network round-trip for data reads. This implies that in the client-server mode, Geode must have a sophisticated fat client library. Geode, in fact provides two such fat client libraries: a Java one and a "native client". Each of these fat client libraries provides a language-level or link-level interface for client programs to communicate via IP networking, to a server (cluster). These fat client libraries provide sophisticated in-process data management.

Two decades ago, the software on which Geode is based, started life as a shared object memory. Multiple processes running on the same computer shared coordinated access to objects through shared memory. In that era there was no client-server mode at all. There was only peer-to-peer. In that world the software's raison d'être was to provide efficient, low-latency, consistent shared object access to processes running on a single computer. Eventually a "caching interface" was added to that system. Some time later the (C-based) shared memory logic was eliminated entirely and a new network-based distributed caching infrastructure took its place. Some of the most significant users of the software on which Geode is based, still run business-critical high-scale applications that rely entirely on this use of the software to support peer-to-peer (cache) data management.

Geode exposes a vast peer-to-peer Java API (e.g. `org.apache.geode.cache.CacheFactory` and friends) that users rely on. While the project has striven to clearly divide the public peer-to-peer API from the internal one (various `..internal..` packages), the existing public API may be too large and too ill-specified to provide the isolation necessary to reasonably consider alternative implementations of that (public, peer-to-peer) interface. This porous peer-to-peer API covers a networking vocabulary of around 360 different peer-to-peer message types.

The Geode client-server Java API (e.g. `org.apache.geode.cache.client.ClientCacheFactory`) is implemented underneath by around 70 network message, or "Op", types. By itself, this client-server interface could possibly serve as an architectural choke-point, were it not for the size, and porosity of the existing peer-to-peer interface.

Decision

Geode is a Java toolkit—not (merely) a server product. While Geode supports a client-server topology, and therefore supports clients written in various programming languages, it also supports a peer-to-peer topology.

This peer-to-peer support is a primary product focus. No effort will be made to end support for this peer-to-peer interface. No effort will be made to abstract a peer-to-peer interface away from the current implementation.

Consequences

The Geode project benefits from no single, cohesive, well-defined client-server choke-point in the architecture.

It would be very difficult to significantly change Geode's server-side (peer-to-peer) architecture. Furthermore, it would be very difficult (and almost nonsensical) to change Geode's server-side implementation language.

Contrast this with client-server products such as PostgreSQL or Redis or RabbitMQ. Few users of those systems know or care what language those servers are written in (C, C, and Erlang respectively). Those projects are much freer (than the Geode project) to change server-side implementation details.

Another contrasting example is Elastic Search, which, like PostgreSQL and RabbitMQ has a clear client-server split and no significant peer-to-peer user API. Elastic Search version 8.0, circa October, 2021 moved to requiring a Java 17 JRE on the server-side. They're still compiling on older JDKs and they support client JREs  back to Java 8 . This is an example where the clear client-server split benefited the Elastic Search project—allowing the project's server-side functionality to skip forward 5 years (from Java 8 to Java 17) without requiring all of their user (client!) applications to move the the newer JRE.

Format

This GAP format is based on Michael Nygard's ADR Template.