THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Note |
|---|
At this time, YARN's Node Manager only works with the CPU subsystem. |
The cgroups Cgroups Linux kernel feature allows aggregating/partitioning a set of tasks and their future children into hierarchical groups with respect to one or more subsystems. For example: When cgroups are enabled for the CPU subsystem, and a task is launched by Mesos Slave, it goes under the following hierarchy:
...
| Code Block | ||
|---|---|---|
| ||
/sys/fs/cgroup/cpu/mesos/<parent-mesos-id>/hadoop-yarn/<child-yarn-id-1> /sys/fs/cgroup/cpu/mesos/<parent-mesos-id>/hadoop-yarn/<child-yarn-id-2> |
Enabling
...
Cgroups for meso-slave
To enable
...
Cgroups for mesos-slave, start the slave with the following flag:
| Code Block | ||
|---|---|---|
| ||
--isolation=cgroups/cpu,cgroups/mem |
Enabling
...
Cgroups for YARN Node Manager
To enable cgroups Enabling Cgroups for YARN NodeManager involves:
- Edit the $YARN_HOME/etc/hadoop/myriad-config-default.yml file.
- Edit the $YARN_HOME/etc/hadoop/yarn-site.xml file.
Modify myriad-config-default.yml
Modify the $YARN_HOME/etc/hadoop/myriad-config-default.yml file by adding the following content:
| Code Block |
|---|
... nodemanager: cgroups: true ... |
Modify yarn-site.xml
Modify the $YARN_HOME/etc/hadoop/yarn-site.xml file by adding the following content:
...