Mesos integration

Design

Apache Mesos Framework support scheduling and running Apache Ignite nodes in a Mesos cluster. This develop a Mesos Scheduler, a Mesos Task.

Goal Scenario

User starts Ignite Mesos Framework, framework scheduler is started and registers in the Mesos Cluster. Once registration is successful the framework will start to launch tasks on nodes. The framework will begin expanding the cluster to the desired total number of nodes. If a node were to be lost, the framework would first attempt to restart the node and the framework will remove the lost node from the cluster.

Components

Mesos Framework

Multiple instances of the framework can be run for multiple topology Frameworks will re-register with Mesos. Fault Tolerant, to restart failed Apache Ignite node tasks.

Mesos Scheduler

The Mesos scheduler is the component with the most high-level intelligence in the framework. It will need to possess the ability to start Apache Ignite cluster and distribute the correct configuration and packages to all started nodes.

Responsibilities

Restarting failed nodes
Providing configuration to nodes (JVM OPTS)
Auto scaling
Providing package to slave nodes. (ignite package, users libs, config file)

Mesos Task

The Mesos Task will be responsible for running container with required configuration.

How run Ignite Framework

1. Build ignite-mesos module.
2. Run IgniteFramework use the followin command.

java -jar ignite-mesos-1.0.0-snapshot-jar-with-dependencies.jar

Make sure that Ignite cluster is running for this:

3. Open Mesos console at http://<master-ip>:5050

4. Go to sandbox

5. Look at stdout and stderr logs.

Properties

The following properties can be used to configure the framework. The properties can be passed to how property of file or environment variables.

name	description	default	example
IGNITE_RUN_CPU_PER_NODE	The number of CPU Cores for each Apache Ignite node.	UNLIMITED	2
IGNITE_MEMORY_PER_NODE	The number of Megabytes of RAM for each Apache Ignite node.	UNLIMITED	1024
IGNITE_DISK_SPACE_PER_NODE	The number of Megabytes of Disk for each Apache Ignite node.	1024	1024
IGNITE_NODE_COUNT	The number of nodes in the cluster.	5	10
IGNITE_TOTAL_CPU	The number of CPU Cores for Ignite cluster.	UNLIMITED	5
IGNITE_TOTAL_MEMORY	The number of Megabytes of RAM for Ignite cluster.	UNLIMITED	16024
IGNITE_TOTAL_DISK_SPACE	The number of Megabytes of Disk for each Apache Ignite cluster.	UNLIMITED	5024
IGNITE_MIN_CPU_PER_NODE	The minimum number of CPU cores required to run Apache Ignite node.	1	4
IGNITE_MIN_MEMORY_PER_NODE	The minimum number of Megabytes of RAM cores required to run Apache Ignite node.	256	1024
IGNITE_VERSION	The version ignite which will be run on nodes.	latest	1.0.5
IGNITE_WORK_DIR	The directory which will be used for saving.	ignite-release	/opt/ignite/
IGNITE_XML_CONFIG	The path to Apache Ignite config file.	N/A	/opt/ignite/ignite-config.xml
IGNITE_USERS_LIBS	The path to libs which will be added to classpath.	N/A	/opt/libs/
MESOS_MASTER_URL	Mesos ZooKeeper URL to locate leading master.	zk://localhost:2181/mesos	zk://176.0.1.45:2181/mesos or 176.0.1.45:2181

Page tree