Versions Compared

Old Version 9

changes.mady.by.user Nikolai Tikhonov

Saved on

compared with

New Version Current

changes.mady.by.user Nikolai Tikhonov

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Design

Apache Mesos Framework should support scheduling and running Apache Ignite nodes in a Mesos cluster. This would develop a Mesos Scheduler, a Mesos Task. Java will be used as the implementation language for the Framework Scheduler.

Goal Scenario

User starts Ignite Mesos Framework, framework scheduler is started and registers in the Mesos Cluster. Once registration is successful the framework will start to launch tasks on nodes. The framework will begin expanding the cluster to the desired total number of nodes. If a node were to be lost, the framework would first attempt to restart the node and if unable to restart the node the framework would will remove the lost node from the cluster.

...

Multiple instances of the framework can be run for multiple topology Frameworks will re-register with Mesos when restarting if it was previously registered
Fault . Fault Tolerant, to restart failed Apache Ignite node tasks.

...

The Mesos scheduler is the component with the most high-level intelligence in the framework. It will need to possess the ability to start Apache Ignite cluster and distribute the correct configuration and packages to all started nodes.

Responsibilities

  • Restarting failed nodes
  • Providing configuration to nodes (JVM OPTS)
  • Auto scaling
  • Providing package to slave nodes. (ignite package, users libs, config file)

Mesos Task 

The Docker Mesos Task will be responsible for running container with required configuration.

Running Ignite Framework

For running Ignite Framework requires started Apache Mesos Cluster. Instruction how setting up a cluster: https://docs.mesosphere.com/getting-started/datacenter/install/.

How run Ignite Framework

...

Run the Framework via Marathon

Currently the recommended way to run the Ignite Framework is via Marathon. 

  1. Install marathon. See https://docs.mesosphere.com/getting-started/datacenter/install/ marathon section.

  2. Download or copy the json properties. Update the MESOS_MASTER_URL and any other parameters which would like to change. See Properties section.

    Code Block
    languagejs
    { 
  "id": "ignition", 
  "instances": 1, 
  "cpus": 2, 
  "mem": 2048, 
  "ports": [0], 
  "uris": [ 
    "https://s3.amazonaws.com/ignite-mesos-ip/ignite-mesos-1.1.0.jar" 
  ], 
  "env": { 
    "IGNITE_NODE_COUNT": "3", 
    "MESOS_MASTER_URL": "zk://localhost:2181/mesos", 
    "IGNITE_RUN_CPU_PER_NODE": "2", 
    "IGNITE_MEMORY_PER_NODE": "2048", 
    "IGNITE_USERS_LIBS_URL": "https://s3.amazonaws.com/ignite-mesos-ip/libs.zip",
    "IGNITE_CONFIG_XML_URL": "https://s3.amazonaws.com/ignite-mesos-ip/default-config.xml",
    "IGNITE_VERSION": "1.0.5"
  }, 
  "cmd": "java -jar ignite-mesos-1.1.0.jar" 
}

  3. POST the config to Marathon by CURL or other tools. 

    curl -X POST -H "Content-type: application/json" --data-binary @marathon.json http://<marathon-ip>:8080/v2/apps/

    Make sure thath Ignite cluster is running, for this:

  4. Open Marathon UI at http://<marathon-ip>:8080. Make sure that exists application with name "Ignition".

 

Run JAR file

  1. Build ignite-mesos module.

...

  1. Run IgniteFramework use the followin command.
       java -jar ignite-mesos-1.0.0-snapshot-jar-with-dependencies.jar 

    Make sure that Ignite cluster is running, for this:

    3. Open Mesos console at http://<master-ip>:5050. If all work correctly tasks with "Ignite node N" should have state "RUNNING" or "STAGING".

Image Added

    4. Go to sandbox.

Image Added

    5. Look at stdout and stderr logs.

Image Addedjar [-cfg (path to config)]

Properties

The following properties can be used to configure the framework. The properties can be passed to how property of file or environment variables.

namedescriptiondefaultexample 
IGNITE_RUN_CPU_PER_NODE
The number of CPU Cores for each Apache Ignite node.
UNLIMITED
2
IGNITE_MEMORY_PER_NODE
The number of Megabytes of RAM for each Apache Ignite node. 
UNLIMITED
1024
IGNITE_DISK_SPACE_PER_NODE
The number of Megabytes of Disk for each Apache Ignite node.
1024
1024
IGNITE_NODE_COUNT
The number of nodes in the cluster.
5
10
IGNITE_TOTAL_CPU
The number of CPU Cores for Ignite cluster.
UNLIMITED
5
IGNITE_TOTAL_MEMORY
The number of Megabytes of RAM for Ignite cluster. 
UNLIMITED
16024
IGNITE_TOTAL_DISK_SPACE
The number of Megabytes of Disk for each Apache Ignite cluster.
UNLIMITED
5024
IGNITE_MIN_CPU_PER_NODE
The minimum number of CPU cores required to run Apache Ignite node.
1
4
IGNITE_MIN_MEMORY_PER_NODE
The minimum number of Megabytes of RAM cores required to run Apache Ignite node.
256
1024
IGNITE_VERSION
The version ignite which will be run on nodes.
latest
1.0.5
IGNITE_WORK_DIR
 The directory which will be used for saving. 
ignite-release
 /opt/ignite/
IGNITE_XML_CONFIG
 The path to Apache Ignite config file.N/A
/opt/ignite/ignite-config.xml
IGNITE_USERS_LIBS
 The path to libs which will be added to classpath.N/A
/opt/libs/
MESOS_MASTER_URL
Mesos ZooKeeper URL to locate leading master.
zk://localhost:2181/mesos

zk://176.0.1.45:2181/mesos or

176.0.1.45:2181

Resources

...

  1. JIRA
  2. Apache Mesos
  3. Setting up a Mesosphere Cluster