Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

~ $ git clone git://github.com/ivankelly/hadoop-common.git
~ $ cd hadoop-common
~/hadoop-common $ mvn package -Pdist -DskipTests

Once compiled, you must configure the namenode.

~/hadoop-common $ export HADOOP_COMMON_HOME=$(pwd)/$(ls d hadoop-common-project/hadoop-common/target/hadoop-common*-SNAPSHOT)
~/hadoop-common $ export HADOOP_HDFS_HOME=$(pwd)/$(ls d hadoop-hdfs-project/hadoop-hdfs/target/hadoop-hdfs*-SNAPSHOT)
~/hadoop-common $ export PATH=$HADOOP_COMMON_HOME/bin:$HADOOP_HDFS_HOME/bin:$PATH

Edit $HADOOP_COMMON_HOME/etc/hadoop/hdfs-site.xml and add the following configuration.

Code Block
xml
xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost/</value>
</property>

<property>
<name>dfs.namenode.name.dir</name>
<value>file:///grid/1/dev/ivank/data/bk-nn-local-snapshotImage Added</value>
</property>

<property>
<name>dfs.namenode.edits.dir</name>
<value>bookkeeper://localhost:2181/hdfsjournal</value>
</property>

dfs.namenode.shared.edits.dir

<property>
<name>dfs.datanode.data.dir</name>
<value>file:///grid/1/dev/ivank/data/bk-dn-local,file:///grid/2/dev/ivank/data/bk-dn-local,file:///grid/3/dev/ivank/data/bk-dn-localImage Added</value>
</property>

<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>

<property>
<name>dfs.namenode.edits.journal-plugin.bookkeeper</name>
<value>org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager</value>
</property>

Code Block
xml
xml

Benchmarks