Versions Compared

Old Version 16

changes.mady.by.user Ruth Harris

Saved on

compared with

New Version 17

changes.mady.by.user Ruth Harris

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • You are using hadoop-2.7.1 downloaded from hadoop.apache.org. Specific vendor versions should work but may require additional steps.Hadoop is installed in the
Note

The default location for $YARN_HOME is /opt/hadoop-2.7.1

...

.

 

Building the Myriad Remote Distribution Bundle

...

Code Block
cp myriad-scheduler/build/libs/*.jar /opt/hadoop-2.7.1/$YARN_HOME/share/hadoop/yarn/lib/
cp myriad-scheduler/src/main/resources/myriad-config-default.yml /opt/hadoop-2.7.1$YARN_HOME/share/hadoop/yarn/lib/

 

Step 3: Put the Myriad Executor Jar File

...

Code Block
frameworkSuperUser: admin              # Must be root or have passwordless sudo on all nodes!
frameworkUser: hduser                  # Should be the same user running the resource manager.
                                       # Must exist on all nodes and be in the 'hadoop' group
executor: 
nodeManagerUri: hdfs://namenode:port/dist/hadoop-2.7.1.tar.gz 
path: hdfs://namenode:port/dist/myriad-executor-runnable-0.0.1.jar
yarnEnvironment: 
YARN_HOME: hadoop-2.7.01                # This should be relative if nodeManagerUri is set

...

Step 5: Configure YARN to use Myriad

Modify the /opt/hadoop-2.7.1$YARN_HOME/etc/hadoop/yarn-site.xml file as instructed in Sample: yarn-site.xml file.

...

Code Block
cd ~
sudo cp -rp /opt/hadoop-2.7.01 .
sudo rm hadoop-2.7.01/etc/hadoop/*.xml
sudo tar -zcpf ~/hadoop-2.7.01.tar.gz hadoop-2.7.01
hadoop fs -put ~/hadoop-2.7.01.tar.gz /dist

 

Getting Started

...