Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

./build.sh -a bigtop -o centos-6 -c hdfs

Build a Hadoop HDFS, Hadoop YARN, and Spark on YARN sandbox image

./build.sh -a bigtop -o debian-8 -c "hdfs, yarn, spark"

Build a Hadoop HDFS and HBase sandbox image

./build.sh -a bigtop -o ubuntu-16.04 -c "hdfs, hbase"

Use --dryrun to skip the build and get Dockerfile and configuration

./build.sh -a bigtop -o ubuntu-16.04 -c "hdfs, hbase" --dryrun

Change the repository of packages

  • Change the repository to Bigtop's nightly centos-6 repo
export REPO=http://ci.bigtop.apache.org:8080/job/Bigtop-trunk-repos/BUILD_ENVIRONMENTS=centos-6%2Clabel=docker-slave-06//ws/output
./build.sh -a bigtop -o centos-6 -c "hdfs, yarn, spark, ignite"

Customize your Big Data Stack

  • Edit site.yaml.template.centos-6_hadoop to create your own prefered stack
cp site.yaml.template.centos-6_hadoop site.yaml.template.centos-6_hadoop_ignite
vim site.yaml.template.centos-6_hadoop_ignite
  • Add ignite in hadoop_cluster_node::cluster_components array and leave the other unchanged
...
hadoop_cluster_node::cluster_components: [hdfs, yarn, ignite]
...
  • Build
./build.sh -a bigtop -o centos-6 -f site.yaml.template.centos-6_hadoop_ignite -t my_ignite_stack

Known issues

Fail to start daemons using systemd

Since systemd requires CAP_SYS_ADMIN, currently any OS using systemd can not successfully started up daemons during image build time.

Daemons can be brought up only if --privileged specified using docker run command. 

Please read the doc here.

Available Sandboxes: https://hub.docker.com/r/bigtop/sandbox/tags/

...