THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- Make sure to grab the repo file:
No Format sudo wget -O /etc/yum.repos.d/bigtop.repo http://www.apache.org/dist/incubator/bigtop/stable/repos/[centos5|centos6|fedora]/bigtop.repo
- This step is optional, but recommended: enable the mirror that is closest to you (uncomment one and only one of the baseurl lines and remove the mirrorlist line): . If the downloads are too slow, try another mirror
No Format sudo vi /etc/yum.repos.d/bigtop.repo
- Browse through the artifacts
No Format yum search hadoop
- Install the full Hadoop stack (or parts of it)
No Format sudo yum install hadoop\* flume\* mahout\* oozie\* whirr\*
...
- Make sure to grab the repo file:
No Format sudo wget -O /etc/zypp/repos.d/bigtop.repo http://www.apache.org/dist/incubator/bigtop/stable/repos/suse/bigtop.repo
- Enable the mirror that is closest to you (uncomment one and only one of the baseurl lines). If the downloads are too slow, try another mirror
No Format sudo vi /etc/zypp/repos.d/bigtop.repo
- Browse through the artifacts
No Format zypper search hadoop
- Install the full Hadoop stack (or parts of it)
No Format sudo zypper install hadoop\* flume\* mahout\* oozie\* whirr\*
...
- Install the Apache Bigtop GPG key
No Format wget -O- http://www.apache.org/dist/incubator/bigtop/stable/repos/GPG-KEY-bigtop | sudo apt-key add -
- Make sure to grab the repo file:
No Format sudo wget -O /etc/apt/sources.list.d/bigtop.list http://www.apache.org/dist/incubator/bigtop/stable/repos/ubuntu/bigtop.list
- Enable the mirror that is closest to you (uncomment one and only one pair of deb/deb-src lines). If the downloads are too slow, try another mirror
No Format sudo vi /etc/apt/sources.list.d/bigtop.list
- Update the apt cache
No Format sudo apt-get update
- Browse through the artifacts
No Format apt-cache search hadoop
- Make sure that you have the latest JDK installed on your system as well. You can either get it from the official Oracle website (http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u29-download-513648.html
) or follow the advice given by your Linux distribution. If your JDK is installed in a non-standard location, make sure to add the line below to the /etc/default/hadoop file
No Format export JAVA_HOME=XXXX
- Install the full Hadoop stack (or parts of it)
No Format sudo apt-get install hadoop\* flume\* mahout\* oozie\* whirr\*
...
- You have the latest JDK installed on your system as well. You can either get it from the official Oracle website (http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u29-download-513648.html
) or follow the advice given by your Linux distribution (e.g. some Debian based Linux distributions have JDK packaged as part of their extended set of packages). If your JDK is installed in a non-standard location,
make sure to add the line below to the /etc/default/hadoop file
to the /etc/default/hadoop file.No Format export JAVA_HOME=XXXX
- Format the namenode
No Format sudo -u hdfs hadoop namenode -format
- Start the necessary Hadoop services. E.g. for the pseudo distributed Hadoop installation you can simply do:
No Format for i in hadoop-namenode hadoop-datanode hadoop-jobtracker hadoop-tasktracker ; do sudo service $i start ; done
- Once your basic cluster is up and running it is a good idea to create a home directory on the HDFS:
No Format sudo -u hdfs hadoop fs -mkdir /user/$USER sudo -u hdfs hadoop fs -chown $USER /user/$USER
- Enjoy your cluster
No Format hadoop fs -lsr / hadoop jar /usr/lib/hadoop/hadoop-examples.jar pi 10 1000
...