Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

There are three four main steps for installing Ambari and HDP using VirtualBox and Vagrant:

  1. Install VirtualBox and Vagrant. (These two steps only need to be done once unless you want to upgrade VirtualBox and/or Vagrant.)
  2. Start one or more VMs, each representing a node in a cluster.
  3. Download, install, and deploy the version of Ambari you wish to use.
  4. From Ambari, deploy the version of HDP you wish to use. 

...

Vagrant is a tool that makes it easier to work with virtual machines. It helps automate the work of setting up, running, and removing virtual machine environments. For our purposes,

Vagrant makes it easy for us to install and run a preconfigured pre-configured cluster environment for running Ambari and the HDP stack.

  1. Download and install VirtualBox from https://www.virtualbox.org/wiki/Downloads.
    Note: as of 4/25/16 there are potential issues running Vagrant doesn't work with the latest version of VirtualBox with the Ambari Vagrant setup. We recommend installing an older (4.x) version of VirtualBox. This Quick Start has been tested on 4.3.34.
  2. Download and install Vagrant from http://downloads.vagrantup.com.
  3. Clone the ambari-vagrant GitHub repository into a convenient folder on your Mac. The repository contains scripts for setting up Ambari virtual machines on several Linux distributions.
    From the command-line:

    Code Block
    git clone https://github.com/u39kun/ambari-vagrant.git
    
  4. Edit the /etc/hosts file on your computer so that you will be able to resolve hostnames for the VMs:

    Code Block
    sudo -s 'cat ambari-vagrant/append-to-etc-hosts.txt >> /etc/hosts'
    
  5. Use the vagrant command to create Create a private key . You will need the key for to use with Ambari.
    The vagrant command displays Vagrant command usage and common commandsinformation, and then it creates a private key as in the file ~/.vagrant.d/insecure_private_key.

    Code Block
    vagrant
    


...

  1.  Change your current directory to ambari-vagrant:

    Code Block
    cd ambari-vagrant
    

    You will see subdirectories for different OS’s.

  2. cd into the OS subdirectory for the OS you wish to use. CentOS 6.4 is recommended, because it is quicker to launch than other OS's.
    The remainder of this Quick Start uses CentOS 6.4. To install and use another version or distribution of Linux, specify the other directory name in place of centos6.4

    Code Block
    cd centos6.4
    

    Important: All VM-focused vagrant commands operate within your current directory. Be sure to run them from the Mac subdirectory associated with the operating system you're using. If you attempt to run a Vagrant command from another directory, it will not find the VM.

  3. Copy the private key into your OS directory:

    Code Block
    cp ~/.vagrant.d/insecure_private_key .
    
  4. (Optional) If you have at least 16 GB of memory on your Mac, consider increasing the amount of memory allocated to the VMs. Edit the following line in Vagrantfile, increasing 3072 to 4096 or more:

    Code Block
    vb.customize ["modifyvm", :id, "--memory", 3072] # RAM allocated to each VM
    
  5. All VMs will have a directory called /vagrant inside the VM. This corresponds to the ambari-vagrant/<os> directory on your local computer. If you have any files to access from within the VM, you can place them in this shared directory.
  6. Start one or more VMs, using the ./up.sh command. Each VM will run one HDP node.
    Three nodes can run well if you have 16GB of RAM on your Mac.

    Code Block
    ./up.sh <# of VMs to launch>
    

    Additional notes:
    - With the default Vagrantfile (in each OS subdirectory) you can specify up to 10 VMs. (If your computer can handle it; you can even add more.)
    - The fully-qualified domain name (FQDN) for each VM will have the format <os-code>[01-10].ambari.apache.org, where <os-code> is c59 (CentOS 5.9), c64 (CentOS 6.4), etc. For example, c5901.ambari.apache.org will be the FQDN for node 01 running CentOS 5.9.
    - The IP address for each VM will have the format 192.168.<os-subnet>.1[01-10], where <os-subnet> is 59 for CentOS 5.9, 64 for CentOS 6.4, etc. For example, 192.168.59.101 will be the IP address for CentOS 5.9 node c5901.

    For example, ./up.sh 3 starts 3 VMs. For CentOS 6.4, the associated hostnames will be c6401, c6402, and c6403.

    Note:  up.sh 3 is equivalent to vagrant up c640{1..3}.

  7. Check the status of your VM(s). The following example shows the results of ./upsh 3}.
    Check the status of your VM(s) for CentOS 6.4:

    Code Block
    hw100:centos6.4 lkg$ vagrant status
    
    Current machine states:
    c6401                     running (virtualbox)
    c6402                     running (virtualbox)
    c6403                     running (virtualbox)
    c6404                     not created (virtualbox)
    c6405                     not created (virtualbox)
    c6406                     not created (virtualbox)
    c6407                     not created (virtualbox)
    c6408                     not created (virtualbox)
    c6409                     not created (virtualbox)
    c6410                     not created (virtualbox)

    (You can use the vagrant init command to initialize your current directory as a Vagrant environment, but this step should already be handled by the preceding steps.)

  8. Next, log into one of your VMs. For example:

    Code Block
    vagrant ssh c6401
    

    From this point onward, this terminal window will operate within the VM until you exit the VM. All commands will go to the VM, not your Mac.
    Tip: Open a second terminal window for submitting commands on your Mac. This will be useful when accessing the Ambari Web UI. You can tell which window you're in by looking at the terminal prompt: your VM will list the OS name; your Mac will list your Mac's computer name.

  9. When you first access the VM you will be logged in as user vagrant. Change to root user:

    Code Block
    sudo su -
    
  10. When you are finished using the VM, TBD (save, exit; snapshot)

...

Timeline for installing HDP, and mac hardware

 

Troubleshooting

 

Other tips:

 

  • yum failed

...

  • ? install yum on your VMs.
  • Ambari message about THPs? fix transparent huge page setting in /etc/rc.loca (CentOS7)

...

  • ntpd not installed?

...

Troubleshooting

...

Basic Vagrant Commands

CommandDescription
vagrant up <vm-name>

Starts a specific VM. (up.sh is a wrapper for this call.) You can use this command to start more VMs after you called up.sh.

Example: vagrant up c6406

Note: if you do not specify the <vm-name> parameter, it will try to start ten VMs.

vagrant status [<vm-name>]Shows which VMs are running, suspended, etc.
vagrant destroy -f [<vm-name>]

Destroys all VMs launched from the current directory, and deletes them from disk.

Optional: Specify a specific VM to destroy.

vagrant suspend [<vm-name>]

Suspends (snapshot) all VMs launched from the current directory so that you can resume them later

Optional: Specify a specific VM to suspend.

vagrant resume [<vm-name>]

Resumes all suspended VMs launched from the current directory

Optional: Specify a specific VM to resume.

vagrant ssh <vm-name>

Starts a SSH session to the host.

Example: vagrant ssh c6401

vagrant --helpList information about Vagrant commands.

...