Page History

...

Build a Hadoop HDFS, Hadoop YARN, and Spark on YARN sandbox image

./build.sh -a bigtop -o debianubuntu-816.04 -c "hdfs, yarn, spark"

Build a Hadoop HDFS and HBase sandbox image

...

Change the repository of packages

...

export REPO=httphttps://ci.bigtop.apache.org:8080/view/Packages/job/Bigtop-trunk-repos/BUILD_ENVIRONMENTSOS=centos-6%2Clabelubuntu-16.04,label=docker-slave-06//ws/output/apt/
./build.sh -a bigtop -o centosubuntu-616.04 -c "hdfs, yarn, spark, ignite"

Customize your Big Data Stack

• Edit site.yaml.template.centos-6_hadoop to create your own prefered stack

cpvim site.yaml.template.centosdebian-68_hadoop site.yaml.template.centos-6_hadoop_ignite
vim site.yaml.template.centos-6_hadoop_ignite

• Add ignite in hadoop_cluster_node::cluster_components array and leave the other unchanged

...
hadoop_cluster_node::cluster_components: [hdfs, yarn, ignite]
...

• Build

# Configure your own stack

./build.sh -a bigtop -o centosdebian-68 -f site.yaml.template.centosdebian-68_hadoop_ignite -t my_ignitehadoop_stack

Known issues

Fail to start daemons using systemd

Since systemd requires CAP_SYS_ADMIN, currently any OS using systemd can not successfully started up daemons during image build time.

...