This how-to briefly explains how to launch a Geode cluster in docker. Specifically, each component (locator and servers) will be running in their own container. The cluster will be accessible (either using gfsh
or through regular Geode client APIs) from the host system. This scenario mimics a typical setup where the cluster is isolated as a backend service and made accessible to a frontend application.
Since the how-to is intended to be developer focused we will not be creating a docker container containing Geode but instead will be bind mounting the Geode distribution into the containers. This allows for quicker iteration when code is updated.
The set up is focused on Docker Desktop for Mac (tested on version 2.5.0) but should also work on Linux or Docker for Windows.
A script is provided which will idempotently create the cluster.
The following is a logical depiction of the final configuration
The containers will be created on a separate bridge network. This has the advantage that all containers on this network will be able to communicate with each other. In addition, all ports will be isolated reducing the possibility of port conflicts with services running on the host. In order to expose access to the outside (i.e. the host system) any relevant ports need to be exposed.
Note
Currently Docker for Mac does not allow the network=host
option which would obviate the need to expose ports.
Both the current working directory as well as the location of the Geode distribution will be mounted within each container under the directories /work
and /geode
respectively. Logs for each member will be written into the working directory. (locator1, server1, server2, etc.)
Step-by-step guide
1. Get Geode
We'll need a distribution of Geode to work with. The easiest is simply to clone the repo and build a distribution that we can use:
$ git clone https://github.com/apache/geode.git $ cd geode $ ./gradlew devBuild installDist
This will produce a distribution in the directory: geode-assembly/build/install/apache-geode
(the distribution is essentially all the jars and scripts that are needed to run geode).
2. Create your Script
Copy the following script into your working directory. This is also where logs for each member will be created.
#!/usr/bin/env bash set -x THIS_SCRIPT=$(basename $0) WORK_DIR=$(cd $(dirname $0); pwd) if [ -z "${GEODE_HOME}" ]; then GEODE_HOME=${PWD}/geode-assembly/build/install/apache-geode fi GFSH=${GEODE_HOME}/bin/gfsh function startServer() { local X=$1 local SERVER_PORT=$2 local DEBUG_PORT=$((5004 + X)) pkill -9 -f "start server${X}" rm -rf server${X} $GFSH start server \ --name=server${X} \ --locator-wait-time=120 \ --hostname-for-clients=localhost \ --server-port=$SERVER_PORT \ --log-level=info \ --locators=locator1[10334] \ --J=-Dgemfire.statistic-archive-file=statistics-${X}.gfs \ --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=${DEBUG_PORT} tail -f server${X}/server${X}.log } function startLocator() { rm -rf locator1 $GFSH start locator \ --name=locator1 \ --J=-Dgemfire.jmx-manager-bind-address=0.0.0.0 \ --J=-Dgemfire.jmx-manager-hostname-for-clients=localhost tail -f locator1/locator1.log } case "$1" in locator) pushd ${WORK_DIR} startLocator popd exit 0 ;; server1) pushd ${WORK_DIR} startServer 1 40404 popd exit 0 ;; server2) pushd ${WORK_DIR} startServer 2 40405 popd exit 0 ;; esac docker stop -t 0 locator1 server1 server2 2>/dev/null docker rm locator1 server1 server2 2>/dev/null docker network rm geode-net 2>/dev/null docker network create geode-net 2>/dev/null DOCKER_IMAGE=bellsoft/liberica-openjdk-debian:11 docker run -d \ --name=locator1 \ -e GEODE_HOME=/geode \ -v ${GEODE_HOME}:/geode \ -v ${PWD}:/work \ --network=geode-net \ --hostname=locator1 \ -p 10334:10334 \ -p 1099:1099 \ geode /work/${THIS_SCRIPT} locator docker run -d \ --name=server1 \ -e GEODE_HOME=/geode \ -v ${GEODE_HOME}:/geode \ -v ${PWD}:/work \ --network=geode-net \ --hostname=server1 \ -p 40404:40404 \ -p 5005:5005 \ geode \ /work/${THIS_SCRIPT} server1 docker run -d \ --name=server2 \ -e GEODE_HOME=/geode \ -v ${GEODE_HOME}:/geode \ -v ${PWD}:/work \ --network=geode-net \ --hostname=server2 \ -p 40405:40405 \ -p 5006:5006 \ geode \ /work/${THIS_SCRIPT} server2
When run, the script will automatically create one locator and 2 servers. When re-run it will clean up existing docker containers, networks and working directories.
At this point you may use the gfsh
utility to connect to the cluster.
Points to Note:
- The default docker image used is
bellsoft/liberica-openjdk-debian:11
. However, any image that provides at least JRE version 8 or greater and abash
shell should work. - Since the script launches all members all at once (locators and servers), the servers have the
--locator-wait-time
option set so that they will wait for the locator to become available. - Once the script completes, the cluster will not immediately be ready and may take a minute or two to complete startup.
- The locator port is exported as
10334
on the host. - Each server has an incrementing debug port exposed on the host (5005, 5006).
- The --
hostname-for-clients
property must be set when starting the servers and point to the system which has server ports exposed. In this example it is simply set tolocalhost
which assumes that all clients, connecting to the cluster, will be running on the host system itself. - In order for
gfsh
to be able to connect to the locator, both thejmx-manager-hostname-for-clients
andjmx-manager-bind-address
properties must be set on the locator. - Since
gfsh
forks the actual java locator or server process (and then exits), thedocker run
process needs to be kept alive otherwise the container would simple exit. This is achieved by simply tailing the relevant log file.
Related articles