Page History

...

./gradlew build

Step 2: Deploy the Myriad Jar and Configuration Files

To deploy Myriad Scheduler and Executor Common, Executor, and Scheduler jar files:

1. Copy the Myriad Scheduler jar files from the $PROJECT_HOME/myriad-scheduler/build/libs/ directory to the $YARN_HOME/share/hadoop/yarn/lib/ directory on all nodes in your cluster. 
2. Copy the Myriad Executor myriad-executor-xxx.jar file from the $PROJECT_HOME/myriad-executor/build/libs/ directory to each mesos slave's $YARN_HOME/share/hadoop/yarn/lib/ directory.

To deploy the Myriad configuration file:

1. Copy the myriad-config-default.yml file from $PROJECT_HOME/myriad-scheduler/build/src/main/resources/ directory to the $YARN_HOME/etc/hadoop directory.

...

// Add following to $YARN_HOME/etc/hadoop/mapred-site.xml:

<!--This option enables dynamic port assignment by mesos -->
<property>
    <name>mapreduce.shuffle.port</name>
    <value>${myriad.mapreduce.shuffle.port}</value>
</property>

Step 5 Create dist Directories and Tar Files

There are two tar files uploaded to Mesos when the ResourceManager is started: (1) configuration–all of the hadoop configuration files in the $HADOOP_CONF_DIR/ and (2) binary–contains all files of the $HADOOP_HOME directory. Both of these files are placed within a dist directory, and the paths to both are specified in the myriad-default-conf.yml file.

In order to prepare and generate the configuration and binary tar files, do the following:

• Navigate to the directory containing the $HADOOP_HOME directory (i.e., /usr/local or /opt) and then generate the binary tar gz file as follows:
  
  

cd /opt
tar czf hadoop.tgz hadoop-2.6.0

• Create a dist directory and a config subdirectory within the $HADOOP_HOME directory, copy the contents of /etc/hadoop into the newly created dist/config directory, and generate the config tar gz file as follows:
  
  

mkdir dist
mkdir dist/config
cp /etc/hadoop/* dist/config
cd dist
tar czf config.tgz config

Starting the Resource Manager

...

./sbin/yarn-daemon.sh start resourcemanager 

Next, creat

Running Myriad Executor and Node Managers

...