Versions Compared

Old Version 13

changes.mady.by.user Ruth Harris

Saved on

compared with

New Version Current

changes.mady.by.user Ruth Harris

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: mistake

...

  • The Mesos Slave processes advertises all of a node’s resources (8 CPUs, 16 GB RAM) to the Mesos Master.
  • The YARN Node Manager is started as a Mesos Task. This task is allotted (4 CPUs and 8 GB RAM) and the Node Manager is configured to only advertise 3 CPUs and 7 GB RAM.

  • The Node Manager is also configured to mount the YARN containers under a cgroup Cgroup hierarchy - which stems from a Mesos task. For example:

    Code Block
    /sys/fs/cgroup/cpu/mesos/node-manager-task-id/container-1

...

  1. Myriad makes a decision to launch a new Node Manager.
    1. Myriad passes the required configuration and task launch information to the Mesos Master which forwards that to the Mesos Slave(s).
    2. Mesos Slave launches Myriad Executor which manages the life cycle of the Node Manager.
    3. Myriad Executor upon launch, configures Node Manager (for example, specifying CPU and memory to advertise, cgroups Cgroups hierarchy, and so on) and then launches it. For example: In the previous following diagram, the Node Manager is allotted 2.5 CPU and 2.5 GB RAM.
  2. The YARN Node Manager, upon startup, advertises configured resources to the YARN Resource Manager. In the following example, 2 CPU and 2 GB RAM are advertised. The rest of the resources are used by the Myriad Executor and Node Manager processes to run.
  3. The YARN Resource Manager can launch containers now, via this Node Manager. The launched containers are mounted under the configured cgroup Cgroup hierarchy.