Page History

...

1. The application is added to the Resource Manager's scheduling pipeline.
   
   • If a Node Manager has a zero profile, the YARN scheduler (for example. FairShareScheduler) does not allocate any application containers.
   • If a Node Manager has a non-zero capacity (low, medium, or high profiles), containers might be allocated for those Node Managers depending on their free capacity.
2. Myriad receives resource offers from Mesos for slave nodes running zero profile Node Managers.
3. The offered resources are projected to the YARN Scheduler as available capacity of the zero profile Node Manager. For example, if Mesos offers (10G memory, 4CPU) for a given node, then the capacity of the zero profile Node Manager running on that node increases to (10G memory, 4 CPU).
4. The YARN Scheduler allocates a few containers for the zero profile Node Manager.
5. • For each allocated container, Myriad spins up a placeholder Mesos task that holds on to Mesos resources as long as the corresponding YARN container is alive. The placeholder tasks are launched in a single shot, corresponding to the containers that YARN allocates.
   • Node Managers become aware of container allocations via YARN's heart-beat mechanism.
   • Myriad ensures that Node Managers are made aware of container allocations only after the corresponding placeholder Mesos tasks are launched.
6. When containers finish, Myriad sends out finished status updates to Mesos for the corresponding placeholder tasks.
7. Mesos takes back the resources from Myriad after receiving a finished status update.

...