The Myriad environment is established by running an initial application for the resource manager. This initial application name is specified in the yarn-site.xml file with the the yarn.resourcemanager.hostname property and the value, <app-name>.<framework>.mesos.
For example, if the value of yarn.resourcemanager.hostname property is rm.marathon.mesos:
rm - The ID given to the Resource Manager when launched using Marathon. Mesos-DNS constructs the Resource Manager hostname using the ID.
marathon - The Myriad framework.
If you are using Marathon, launch Marathon and run an initial Resource Manager application. The Resource Manager can be launched or stopped from either the command line or the Marathon UI.
Launching from the Command Line
To start the Resource Manager, run the YARN daemon from the command line:
yarn-daemon.sh start resourcemanager
To shut down he Resource Manager, run the YARN daemon from the command line:
yarn-daemon.sh stop resourcemanager
Launching from Marathon
Alternatively, start and stop Myriad from the Marathon UI. See Marathon: Application Basics for more information. For example, create an application to start the Resource Manager:
cd hadoop-2.7.1/sbin && yarn-daemon.sh start resourcemanager
Alternatively, when launching the Resource Manager in an HA environment, specify value for the yarn.resourcemanager.hostname
property. The hostname is the ID field specified when launching a Marathon application.
To initially launch the Resource Manager from Marathon:
Launch Marathon with <host>:8080. For example: http://10.10.100.16:8080
Click on New App.
Create a new application for the resource manager and specify:
ID
CPU
Memory
Instances
Command.
For example
Parameter | Example Value | Description |
---|---|---|
ID | rm | The ID for the Resource Manager when launched using Marathon. Mesos-DNS constructs the Resource Manager hostname using the ID. |
CPU | 0.2 | Amount of CPU allocated. |
Memory | 2048 | Amount of memory allocated |
Instances | 1 | Number of instances to be launched. |
Command | env && && yarn resourcemanager | Command to launch the resource manager. |
If the yarn.resourcemanager.hostname property is not specified in the yarn-site.xml file, then the -DYARN_RESOURCEMANAGER_OPTS option must be specified. When the -Dyarn.resourcemanager.hostname
property is specified when launching Resource Manager, Myriad propagates this value to the Node Managers launched via the flexup API. The Node Managers can discover back the Resource Manager using the hostname specified in -
Dyarn.resourcemanager.hostname
. If that hostname happens to be a Mesos DNS hostname, then when the Resource Manager moves to another node during failover, Node Managerss can still connect to the new Resource Manager using the same hostname. Marathon helps move the Resource Manager to a different node whenever it detects the failure of Resource Manager.
For example
env && export YARN_RESOURCEMANAGER_OPTS=-Dyarn.resourcemanager.hostname=rm.marathon.mesos && yarn resourcemanager
Some applications might require the yarn.resourcemanager.hostname
property to be explicitly specified as a command line option.