Page History

...

Myriad allows Mesos and YARN to co-exist and share resources with Mesos as the resource manager for the datacenterdata center. Sharing resources between these two resource allocation systems improves overall cluster utilization and avoids statically partitioning resources amongst two separate clusters and resource managers.

The following diagram shows two resource managers running independently which results in a statically partitioned datacenterdata center:

Advertising Resources: Mesos Slave and YARN’s Node Manager

The Mesos Slave and YARN’s Node Manager are processes that run on the host OS. Both processes advertise available resources to Mesos Master and YARN’s Resource Manager respectively. Each process can be configured to advertise a subset of resources. This ability is leveraged, in conjunction with cgroups, to allow Mesos Slave and YARN’s Node Manager to co-exist on a node.

• The

...

• Mesos Slave

...

Image Removed

• Mesos Slave processes advertises all of a node’s resources (8 CPUs, 16 GB RAM) to the Mesos Master.
• The YARN Node Manager is started as a Mesos Task. This task is allotted (4 CPUs and 8 GB RAM) and the Node Manager is configured to only advertise 3 CPUs and 7 GB RAM.
• The Node Manager is also configured to mount the YARN containers under the cgroup hierarchy  which stems from a Mesos task.

The following diagram shows a node running YARN Node Manager as a Mesos Slave task.

Image Added

...