Quick Intro. slide
Here's a 17 pages slide to quickly go through the idea of the BigTop VM and Docker provisioner.
Getting Started
Clone the BigTop source code
$ git clone https://github.com/apache/bigtop.git
To deploy a cluster with the latest Bigtop 1.2.0 (docker based):
- git checkout origin/branch-1.2
- ./gradlew -Pconfig=config_ubuntu_xenial.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 16.04)
- ./gradlew -Pconfig=config_debian8.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
- ./gradlew -Pconfig=config_centos6.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
- ./gradlew -Pconfig=config_centos7.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 7)
- ./gradlew docker-provisioner-destroy (to remove the containers)
To deploy a cluster with the latest Bigtop 1.1.0 (docker based):
- git checkout origin/branch-1.1
- ./gradlew -Pconfig=vagrantconfig_ubuntu.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 14.04)
- ./gradlew -Pconfig=vagrantconfig_debian.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
- ./gradlew -Pconfig=vagrantconfig.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
- ./gradlew docker-provisioner-destroy (to remove the containers)
For Bigtop 1.0.0:
To create a VM based BigTop
...
Hadoop cluster:
$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-vm
$ vim vagrantconfig.yaml
# Update the repo to be:
CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64
And update the component you'd like to deploy, for example:
components: [hadoop, yarn, ignite-hadoop] (effective values)
$ vagrant up
# If using CentOS 7, you might need to disable firewall on each of the machines
$ vagrant ssh <machine_name, e.g. bigtop1>
$ sudo systemctl disable firewalld
$ sudo systemctl stop firewalld
DONE!
To create a Docker based BigTop
...
Hadoop cluster:
$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-docker
$ vim vagrantconfig.yaml
# Update docker image file to point to one of:
bigtop/deploy:centos-6
bigtop/deploy:debian-8
# Update the repo to be:
CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64
And update the component you'd like to deploy, for example:
components: [hadoop, yarn, ignite-hadoop] (effective values)
$ ./docker-hadoop.sh --build-image --create 3
DONE!
Destroy clusters:
destroy VM cluster:
$ vagrant destroy -f
...
$ ./docker-hadoop.sh --destroy
Demo video
Widget Connector | ||||||
---|---|---|---|---|---|---|
|
Enable Port Forwarding
Widget Connector | ||||||
---|---|---|---|---|---|---|
|