...

1. You have the latest JDK installed on your system as well. You can either get it from the official Oracle website (http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u29-download-513648.html) or follow the advice given by your Linux distribution (e.g. some Debian based Linux distributions have JDK packaged as part of their extended set of packages). If your JDK is installed in a non-standard location, make sure to add the line below to the /etc/default/hadoop file

   No Format
   export JAVA_HOME=XXXX

2. Format the namenode

   No Format
   sudo -u hdfs hadoop namenode -format

3. Start the necessary Hadoop services. E.g. for the pseudo distributed Hadoop installation you can simply do:

   No Format
   for i in hadoop-namenode hadoop-datanode hadoop-jobtracker hadoop-tasktracker ; do sudo service $i start ; done

4. Once your basic cluster is up and running it is a good idea to create a home directory on the HDFS:

   No Format
   sudo -u hdfs hadoop fs -mkdir /user/$USER sudo -u hdfs hadoop fs -chown $USER /user/$USER

5. Enjoy your cluster

   No Format
   hadoop fs -lsr / hadoop jar /usr/lib/hadoop/hadoop-examples.jar pi 10 1000

Running Hadoop Components

...

One of the advantages of Bigtop is the ease of installation of the different Hadoop Components without having to hunt for a specific Hadoop Component distribution and matching it with a specific Hadoop version. T

Running Pig

#Install Pig

sudo apt-get install pig

#Create Create a tab delimited file using a text editor and import it into HDFS. Start the pig shell and verify a load and dump work. Make sure you have a space on both sides of the = sign. The statement 'using PigStorage('\t')' tells Pig the columns in the text file are delimited using tabs.

$pig

grunt>A

...

=

...

load

...

'/pigdata/PIGTESTA.txt'

...

using

...

PigStorage('\t');

...

grunt>dump

...

A

...

Running

...

HBase

sudo apt-get install hbase\*

...

sudo service hbase-master start

...

hbase shell

...

# test the HBase shell by creating a HBase table and verifying the table exists in HBase

...

create 't1','f1','f2','f3'
list

1. you should see a verification from HBase the table t1 exists

...