Versions Compared

Old Version 3

changes.mady.by.user Ruth Harris

Saved on

compared with

New Version 4

changes.mady.by.user Ruth Harris

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
sudo mkdir -p /usr/local/libexec/mesos/
sudo cp /vagrant/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar /usr/local/libexec/mesos/
sudo chown hduser:hadoop -R /usr/local/libexec/mesos/

 

Step 4: Configure

...

Myriad

...

Code Block
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>${nodemanager.resource.cpu-vcores}</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>${nodemanager.resource.memory-mb}</value>
</property>
<!-- Configure Myriad Scheduler here -->
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler</value>
<description>One can configure other scehdulers as well from following list: com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler, com.ebay.myriad.scheduler.yarn.MyriadFifoScheduler</description>
</property>

...

Defaults

To configure Myriad itself, update $YARN_HOME/etc/hadoop/myriad-config-default-config.yml with the following content:

Code Block
mesosMaster: 10.141.141.20:5050
checkpoint: false
frameworkFailoverTimeout: 43200000
frameworkName: MyriadAlpha
nativeLibrary: /usr/local/lib/libmesos.so
zkServers: localhost:2181
zkTimeout: 20000
profiles:
small:
cpu: 1
mem: 1100
medium:
cpu: 2
mem: 2048
large:
cpu: 4
mem: 4096
rebalancer: true
nodemanager:
jvmMaxMemoryMB: 1024
user: hduser
cpus: 0.2
cgroups: false
executor:
jvmMaxMemoryMB: 256
path: file://localhost/usr/local/libexec/mesos/myriad-executor-runnable-0.0.1.jar

Step 5: Configure YARN to use Myriad

To configure YARN to use Myriad, update $YARN_HOME/etc/hadoop/yarn-site.xml with following content:

Code Block
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>${nodemanager.resource.cpu-vcores}</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>${nodemanager.resource.memory-mb}</value>
</property>
<!-- Configure Myriad Scheduler here -->
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler</value>
<description>One can configure other scehdulers as well from following list: com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler, com.ebay.myriad.scheduler.yarn.MyriadFifoScheduler</description>
</property>

 

Getting Started

To launch Myriad, run following:

Code Block
 sudosudo su hduser
yarn-daemon.sh start resourcemanager

...