Myriad Use Cases

The main uses cases for Myriad include:

* Containerized enterprise environments
* Web-scale infrastructure consolidation
* Hadoop service provider

Containerized Enterprise Environment

This use case is for environments that are either running separate clusters for Mesos and YARN or are statically partitioning their servers to run both simultaneously.

Key characteristics include:

• Diverse Docker-ized applications on Mesos. 
• Hadoop jobs or applications run on YARN. 
• Workload prioritization by YARN FairScheduler. 
• File access is granted based on the user that submitted a YARN application. 
• Dynamic on-demand allocation of resources to all user application running across both frameworks which ensures full utilization of resources without administrators being a critical path in application provisioning. 
• Fair allocation of resource between Myriad and other frameworks. 
• Isolation where applications such as YARN AMs or Mesos frameworks are isolated from other applications in terms of cpu, memory, and disk space. 
• Shared single data layer. 
• Security where a user running a job is only allowed access to that data and only authorized user can provision Myriad-RM instances. 
• Multiple YARN clusters on the same infrastructure, for example, to host dev, test, and production clusters. Typically tens or hundreds of nodes. 
• Marathon implementation for integration with the PaaS frameworks on Mesos. 
• Spark and Storm application compatibility with both YARN and Mesos. 
• Monitoring through Marathon health checks, Ganglia, and Job History interfaces such as MapReduce and Spark HistoryServer.

Web-scale Infrastructure Consolidation

This use case is similar to the containerized enterprise environment.

Key characteristics include:

• Customized engineering of the data center infrastructure where early stage software may be
  extended, maintained, and integrated with custom tools.
• Very large cluster need to be supported. Typically, thousands of nodes.
• Small number of applications where the scale for each application is very large.
  For example, hundreds of servers performing web front-end tasks.
• Single tenancy supporting a single product.
• Minimal security where security is often enforced by IP firewalls rather than at the application level.

Hadoop Service Provider

This use case is for multi-tenancy environments where each tenant is a different company. Thus, isolation and security is extremely critical. Because of application diversity, is is not likely that non-Hadoop tasks will be run on Mesos. In this case, Mesos is only used to negotiate with per-tenant YARN clusters for resource allocation.

Key characteristics include:

• Isolation with Docker because different customers have different job or application dependencies and Docker packaging is needed to wrap those dependencies with their job or application and because the resources between customers need to be isolated so the resource can not be seen or stolen. 
• Very large YARN clusters need to be supported. Potentially, thousands of nodes. 
• Dynamic flexing so that unused resources are available for other customers because resources can be expensive.