There are three four main steps for installing Ambari and HDP using VirtualBox and Vagrant:
- Install VirtualBox and Vagrant. (These two steps only need to be done once unless you want to upgrade VirtualBox and/or Vagrant.)
- Start one or more VMs, each representing a node in a cluster.
- Download, install, and deploy the version of Ambari you wish to use.
- From Ambari, deploy the version of HDP you wish to use.
...
Vagrant is a tool that makes it easier to work with virtual machines. It helps automate the work of setting up, running, and removing virtual machine environments. For our purposes,
Vagrant makes it easy for us to install and run a preconfigured pre-configured cluster environment for running Ambari and the HDP stack.
- Download and install VirtualBox from https://www.virtualbox.org/wiki/Downloads.
Note: as of 4/25/16 there are potential issues running Vagrant doesn't work with the latest version of VirtualBox with the Ambari Vagrant setup. We recommend installing an older (4.x) version of VirtualBox. This Quick Start has been tested on 4.3.34. - Download and install Vagrant from http://downloads.vagrantup.com.
Clone the
ambari-vagrant
GitHub repository into a convenient folder on your Mac. The repository contains scripts for setting up Ambari virtual machines on several Linux distributions.
From the command-line:Code Block git clone https://github.com/u39kun/ambari-vagrant.git
Edit the
/etc/hosts
file on your computer so that you will be able to resolve hostnames for the VMs:Code Block sudo -s 'cat ambari-vagrant/append-to-etc-hosts.txt >> /etc/hosts'
Use the
vagrant
command to create Create a private key . You will need the key for to use with Ambari.
Thevagrant
command displays Vagrant command usage and common commandsinformation, and then it creates a private key as in the file~/.vagrant.d/insecure_private_key
.Code Block vagrant
...
Change your current directory to
ambari-vagrant
:Code Block cd ambari-vagrant
You will see subdirectories for different OS’s.
cd
into the OS subdirectory for the OS you wish to use. CentOS 6.4 is recommended, because it is quicker to launch than other OS's.
The remainder of this Quick Start uses CentOS 6.4. To install and use another version or distribution of Linux, specify the other directory name in place of centos6.4Code Block cd centos6.4
Important: All VM-focused
vagrant
commands operate within your current directory. Be sure to run them from the Mac subdirectory associated with the operating system you're using. If you attempt to run a Vagrant command from another directory, it will not find the VM.Copy the private key into your OS directory:
Code Block cp ~/.vagrant.d/insecure_private_key .
(Optional) If you have at least 16 GB of memory on your Mac, consider increasing the amount of memory allocated to the VMs. Edit the following line in
Vagrantfile
, increasing 3072 to 4096 or more:Code Block vb.customize ["modifyvm", :id, "--memory", 3072] # RAM allocated to each VM
- All VMs will have a directory called
/vagrant
inside the VM. This corresponds to theambari-vagrant/<os>
directory on your local computer. If you have any files to access from within the VM, you can place them in this shared directory. Start one or more VMs, using the
./up.sh
command. Each VM will run one HDP node.
Three nodes can run well if you have 16GB of RAM on your Mac.Code Block ./up.sh <# of VMs to launch>
Additional notes:
- With the defaultVagrantfile
(in each OS subdirectory) you can specify up to 10 VMs. (If your computer can handle it; you can even add more.)
- The fully-qualified domain name (FQDN) for each VM will have the format<os-code>[01-10].ambari.apache.org
, where<os-code>
isc59
(CentOS 5.9),c64
(CentOS 6.4), etc. For example,c5901.ambari.apache.org
will be the FQDN for node 01 running CentOS 5.9.
- The IP address for each VM will have the format192.168.<os-subnet>.1[01-10]
, where<os-subnet>
is59
for CentOS 5.9,64
for CentOS 6.4, etc. For example,192.168.59.101
will be the IP address for CentOS 5.9 nodec5901
.
For example,./up.sh 3
starts 3 VMs. For CentOS 6.4, the associated hostnames will bec6401
,c6402
, andc6403
.
Note:up.sh 3
is equivalent tovagrant up c640{1..3}
.Check the status of your VM(s). The following example shows the results of
./upsh 3}
.
Check the status of your VM(s) for CentOS 6.4:Code Block hw100:centos6.4 lkg$ vagrant status Current machine states: c6401 running (virtualbox) c6402 running (virtualbox) c6403 running (virtualbox) c6404 not created (virtualbox) c6405 not created (virtualbox) c6406 not created (virtualbox) c6407 not created (virtualbox) c6408 not created (virtualbox) c6409 not created (virtualbox) c6410 not created (virtualbox)
(You can use the
vagrant init
command to initialize your current directory as a Vagrant environment, but this step should already be handled by the preceding steps.)Next, log into one of your VMs. For example:
Code Block vagrant ssh c6401
From this point onward, this terminal window will operate within the VM until you exit the VM. All commands will go to the VM, not your Mac.
Tip: Open a second terminal window for submitting commands on your Mac. This will be useful when accessing the Ambari Web UI. You can tell which window you're in by looking at the terminal prompt: your VM will list the OS name; your Mac will list your Mac's computer name.When you first access the VM you will be logged in as user
vagrant
. Change toroot
user:Code Block sudo su -
- When you are finished using the VM, TBD (save, exit; snapshot)
...
Timeline for installing HDP, and mac hardware
Troubleshooting
Other tips:
- yum failed
...
- ? install yum on your VMs.
- Ambari message about THPs? fix transparent huge page setting in /etc/rc.loca (CentOS7)
...
- ntpd not installed?
...
Troubleshooting
...
Basic Vagrant Commands
Command | Description |
---|---|
vagrant up <vm-name> | Starts a specific VM. ( Example: Note: if you do not specify the |
vagrant status [<vm-name>] | Shows which VMs are running, suspended, etc. |
vagrant destroy -f [<vm-name>] | Destroys all VMs launched from the current directory, and deletes them from disk. Optional: Specify a specific VM to destroy. |
vagrant suspend [<vm-name>] | Suspends (snapshot) all VMs launched from the current directory so that you can resume them later Optional: Specify a specific VM to suspend. |
vagrant resume [<vm-name>] | Resumes all suspended VMs launched from the current directory Optional: Specify a specific VM to resume. |
vagrant ssh <vm-name> | Starts a SSH session to the host. Example: |
vagrant --help | List information about Vagrant commands. |
...