...
Code Block |
---|
vagrant up |
At this point , the VM has will have a single node Mesos cluster running and a single node mesos cluster running.
HDFS cluster running. You can modify the Mesos and Hadoop versions by editing the
corresponding variables in the `Vagrantfile`.
Check that Mesos and Hadoop are running with a browser:
Hadoop/HDFS namenode: http://10.141.141.20:50070
Mesos Master: http://10.141.141.20:5050/
To ssh into the cluster, run the following:
Code Block |
---|
vagrant ssh |
The password for the vagrant user is vagrant.
Setting up YARN/Hadoop
To setup YARN/Hadoop inside VM, run the following YARN setup shell files:
1. Run the first YARN setup shell command from the vagrant directory to create a user hduser in group hadoop. Be sure to remember the password that you provide for this user.
Code Block |
---|
cd /vagrant
./setup-yarn-1.sh |
2. Run the second YARN setup shell command as sudo.
Code Block |
---|
sudo su - hduser
cd /vagrant
./setup-yarn-2.sh |
The following processes should be running:
Code Block |
---|
9844 Jps
6709 NameNode
6393 JobHistoryServer
6874 DataNode |
Note |
---|
Process IDs are different. |
Installing Myriad
Installing Myriad involves downloading the Myriad binaries from GitHub. The Myriad location is currently at https://github.com/mesos/myriad.
To install Myriad:
- Create a new myriad directory.
- Clone the Myriad GitHub.
- Pull the contents to the new directory.
You can switch to the hadoop user when needed to launch hadoop processes
Code Block |
---|
sudo su - hduser |
Configuring Myriad
To configure Myriad:
...
Note |
---|
If a build failure occurs, the issue is not with the build itself, but a failure to write to disk. This can happen when you build outside the vagrant instance first. To resolve this issue, exit the user hduser (type: exit) and build again as the vagrant user. |
Step 2: Deploy the Myriad Files
...
Code Block |
---|
/vagrant/myriad-scheduler/build/libs/myriad-executor-0.1.0.1.jar |
For example, the files are copied to the following locations:
Code Block |
---|
cp /vagrant/myriad-scheduler/build/libs/* $YARN_HOME/share/hadoop/yarn/lib/ cp /vagrant/myriad-executor/build/libs/myriad-executor-0.1.0.1.jar $YARN_HOME/share/hadoop/yarn/lib/ cp /vagrant/myriad-scheduler/src/main/resources/myriad-config-default.yml $YARN_HOME/etc/hadoop/ |
...
Code Block |
---|
mesosMaster: <mesos Master IP address>:5050 checkpoint: false frameworkFailoverTimeout: 43200000 frameworkName: MyriadAlpha nativeLibrary: /usr/local/lib/libmesos.so zkServers: localhost:2181 zkTimeout: 20000 profiles: small: cpu: 2 mem: 2048 medium: cpu: 4 mem: 4096 large: cpu: 10 mem: 12288 rebalancer: true nodemanager: jvmMaxMemoryMB: 1024 user: hduser cpus: 0.2 cgroups: false executor: jvmMaxMemoryMB: 256 path: file://localhost/usr/local/libexec/mesos/myriad-executor-runnable-0.1.0.1.jar |
Step 4: Configure YARN to use Myriad
...
To launch Myriad, run the following:
Code Block |
---|
sudo su - hduser
yarn-daemon.sh start resourcemanager |
...
Code Block |
---|
yarn-daemon.sh stop resourcemanager
./vagrant/shutdown.sh
exit
exit
vagrant halt |