Overview
This article serves to dive deeper into the internal architecture of two of the Geode session management modules - the Tomcat session module and TC Server session module. The third session state module is the generic AppServer session module, but the use cases of this module are dwindling as most can be addressed using Spring Session for Apache Geode. There is still enough demand for the Tomcat and TC Server session modules to justify documenting their internals. It is important to note that the Tomcat and TC session modules share the exact same jars, and the only difference between them is configuration and setup scripts. As such, for the remainder of this article, we will just refer to the Tomcat module, but all of this analysis also applies to TC Server module.
As discussed in the documentation, the installation of the Tomcat module is relatively straight forward from a user point of view, and simply involves putting the required jars on the class path, making a handful of configuration changes, and potentially starting a Geode server (only if using the client-server topology). At a high level, the configuration changes inform Tomcat to use the Geode session state module, and the jars contain the Geode-specific implementation of Tomcat specific interfaces and classes which are required of any Tomcat session module, as well as supporting classes/interfaces. The Tomcat session module ultimately adheres to the Java servlet HTTP session specification.
Architecture
The Geode session module for Tomcat contain code which implements specific interfaces and classes expected of any Tomcat session module. Geode aims to provide a session state implementation which uses delta propagation to reduce the amount of data communicated to the Geode server. As such, Geode overrides the StandardSession class provided by Tomcat in addition to the delta propagation related interfaces (Delta, GatewayDelta). We also introduced a session-specific interface for delta propagation, DeltaSessionInterface, which contains methods for updating individual attributes, committing or aborting changes, amongst other functionality. It is important to note that this is a custom Geode implementation of delta functionality, which is distinct from the Tomcat provided DeltaSession class. We have separate DeltaSession classes for each supported version of Tomcat - 7, 8, and 9.
These DeltaSession objects are instantiated by a DeltaSessionManager, which inherits from the Tomcat ManagerBase class and Manager interface. This component manages the access, lifecycle, and persistence of Session objects. The DeltaSessionManager creates and initializes the SessionCache object, which is used to store the Session objects. It also registers two Tomcat Valve implementations - one for committing valid sessions and another for handling session failover. More details on the SessionCache and Valve implementations are discussed below.
The SessionCache provides access to the Geode Cache and Region used to store the Session objects. This includes adding, removing, and "touching" sessions (resetting their TTL by performing a get on the region entry). This component abstracts away most of the actual interaction with the Geode specific components from the SessionManager (with the exception of session persistence to disk). Most of the logic for managing sessions is in the AbstractSessionCache base class, but depending on the chosen topology either a ClientServerSessionCache or a PeerToPeerSession class will be used. A summary of the differences between these two SessionCache implementations are outlined below:
ClientServerSessionCache
- "Bootstraps" the servers with functions for creating the server session region, "touching" sessions (updating TTL), and retrieving the session region size
- Creates and validates the server session region via server-side function execution
- Creates the local session region using ClientRegionFactory
- "Touches" sessions (performing a get on the region entry) to update TTL using onServers for replicated regions or onRegion for partitioned regions
PeerToPeerSessionCache
- Registers functions for "touching" sessions (updating TTL)
- Creates and validates the server session region
- "Touches" sessions (performing a get on the region entry) to update TTL using onMembers for replicated regions or onRegion for partitioned regions
The following class model diagram shows all of the server-side functions used in the Tomcat session state module. Here is a short description of each:
- BootstrappingFunction (Client-Server topology only): Responsible for registering the other functions (hence bootstrapping). Namely, it registers the two TouchX functions, the CreateRegion function, and the RegionSize function. This function uses a MembershipListener to bootstrap any new servers that join the cluster.
- RegionSizeFunction: Returns the current number of entries in the session region
- TouchReplicatedRegionEntriesFunction: Performs get operations on the provided sessions to update their TTL. This ensures we update TTL on when the a read (getAttribute) is performed on the session. Without this function, we would prematurely expire/remove the session if only reads were being performed by a client.
- TouchPartitionedRegionEntriesFunction: Same as above, but used if the session region is partitioned rather than replicated.
- CreateRegionFunction (Client-Server topology only): Used to create the session region on the servers in a client-server topology.
Earlier it was mentioned that the SessionManager registers two implementations of the Tomcat Valve interface. The Valve interface allows us to plug into request pipeline to perform custom logic. Below is a short description of each:
- CommitSessionValve: Invokes logic at the end of the pipeline which will commit the session associated with the request if it is still valid. The validity of the session is determined by querying the Tomcat StandardSession class, and is not Geode-specific. Successful validation results in a commit, which in Geode translates to a put to the session region with the updated session.
- JvmRouteBinderValve: Handles session failover if the requested JVM route does not match the sessions JVM route.
The Tomcat session module registers several CacheCallback implementations for various reasons. Below is a description of each registered Listener:
- GatewayDeltaEventApplicationCacheListener (Unused/Partially Implemented): Applies remote gateway session events to the session region
- GatewayDeltaForwarderCacheListener (Unused/Partially Implemented): Creates events to be applied to the gateway delta region
- DebugCacheListener: Used to log CacheListener events for debugging
- RegionConfigurationCacheListener: Part of the CreateRegionFunction internals. Attached to a region for storing region configuration metadata, and it will create a new session region when a configuration update is received.
- SessionExpirationCacheListener: Listens for region expiration events and in turn invoke the Tomcat StandardSession expiration logic
- SessionCustomExpiry: Translates session "max inactive interval" time, as specified by Tomcat configuration, to region entry TTL/expiration
- LocalSessionCacheWriter (Unused/Partially Implemented): Was supposed to be used when separate "fronting" and "backing" regions were supported in peer-to-peer topology, but this appears to not be used/supported.
- LocalSessionCacheLoader: Same as above
Several helper classes are used in the Tomcat session module. Here are descriptions of each:
- RegionHelper: Used in a peer-to-peer topology to assist in creating and validating the session region
- ResourceManagerValidator: Currently only used to validate JVM arguments related to the session module's use of the Geode ResourceManager for managing heap
- LifecycleTypeAdapter: Used to adapt Catalina style lifecycle events to the generic AppServer module.
- RegionConfiguration: Stores the session region's configuration
- DeltaSessionFacade: Delegates specific calls to the underlying DeltaSession
- DeltaSessionStatistics: Tracks the number of sessions created, invalidated, or expired
- ModuleStatistics (Unused): Tracks cache hits and misses
- RegionStatus: Enum for region validity after creation
- ContextMapper: Provides a mapping from Tomcat Context object to SessionManager. This handles a specific edge case outlined in the Javadoc for this class
- Banner: Prints the session module banner for use in logging