Checking out

...

The following uses Ambari git for the development process. Please keep in mind git is read only and patches will have to be uploaded and committed via svn.

git clone git://git.apache.org/ambari.git

git checkout -b AMBARI-666 origin/AMBARI-666

Ambari source

Follow the instructions under Checkout source code section of "How to contribute" guide.

We'll refer to the top-level "ambari" directory as AMBARI_DIR in this document.

Building Ambari

After checking out Ambari, you can build ambari with the steps below:

$ mvn clean install

Note: The above is for just building the jars. See below for starting/deploying the server and the agent.

Building Artifacts and Running the Server/Agent

NOTE: Though the documentation below mentions agent its not there yet and is coming up.

To build tarball artifacts run:

$ mvn clean package assembly:assembly

This will build the server/agent tarballs.

NOTE: Run everything as root below. This will change as we go forward but for now run everything as root.

For running the server:

$ cp ambari-server/target/ambari-server-*-dist.tar.gz INSTALL_DIR /* where install dir is some directory where you want to run the server */

$ cd INSTALL_DIR

$ tar -zxvf ambari-server-*-dist.tar.gz

$ vim /etc/ambari/ambari.properties
Add the following entry:

security.server.keys_dir=INSTALL_DIR/ambari-server-1.0.3-SNAPSHOT/keystore

NOTE: INSTALL_DIR will have to be replaced with the actual path in the config file.

$ cd ambari-server-*
$ export AMBARI_PASSPHRASE="tmp"
$ java -cp etc/ambari-server/conf/:lib/ambari-server/* org.apache.ambari.server.controller.AmbariServer

NOTE: conf should come first in the java classpath. Unfortunately, guice-persist.jar includes a log4j.properties which causes our log4j.properties to not load.

The server will come up and run as a foreground process. Scripting will come later as we move along to run the server with proper confs/envs.

To check if the server is up and healthy, you can go to:

http://HOSTNAME:8080/api/check^{Image Removed}

The status will show as running.

or you can do: curl localhost:8080/api/check

NOTE: The health check is static as of now and will change later.

Running the agent.

The agent requires python 2.6 to run on a host. RHEL5/Centos5 have python 2.4 installed by default. To install 2.6 (NOTE: you can install both 2.4/2.6 on the host), install the epel repo and then run

$ yum install python26.

Python needs setuptools to be able to setup the project for development. Download http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086^{Image Removed} and then run:

$ sh setuptools-0.6c11-py2.6.egg

To setup the ambari agent:

$ mkdir -p /var/ambari /var/run/ambari /var/log/ambari /tmp/ambari-agent/

1. The following dev process is easier than using setup.py and is a quicker turnaround. As soon as we have a good deployment story (rpm) we will add those in.

$ cd ambari-agent/src/main/python/ambari_agent

1. Note that ambari agent requires facter to get the node information. Download facter tarball from http://downloads.puppetlabs.com/facter/facter-1.6.0.tar.gz^{Image Removed} and uninstall in a directory.

Edit facter_home in AmbariConfig.py to the directory where facter is installed. The agent executes the script facter_home/bin/facter to get all the node information and also uses df to get the mount point information.

To run the agent:
$ export AMBARI_PASSPHRASE="tmp"
$ python2.6 main.py

The Server currently doesnt implement the apis needed for the agent, so the agent will just try connecting to the server and do nothing. Take a look at

/var/log/ambari.log to see the logs.

Building and running Ambari Web

Ambari Web uses Brunch (http://brunch.io/) as the application assembler for development purposes.
Brunch is used to build and deploy automatically in the background as the developer modifies source files (JS, LESS/CSS, Handlebars template files).
It can also start up a Node.js-based web server with a single command so that you can easily run the application from the application root directory without any special configuration.

To build, deploy, and run Ambari Web:

1. Install NPM per http://nodejs.tchol.org/

2. Install Brunch
$ sudo npm install -g brunch

3. Download dependent packages into ambari-web/node_modules directory
$ cd AMBARI_DIR/ambari-web
$ npm install

To run ambari-web so that it works with mock data independently from ambari-server:
$ brunch watch --server (or simply "brunch w --s")
Hit http://localhost:3333^{Image Removed} from your web browser

To run ambari-web with ambari-server integration
  $ cd AMBARI_DIR
  $ mvn clean package -Dweb \[-DskipTests=true\]
  Note that currently the mvn command does not put ambari-web in the Maven-generated package.  This will be fixed in the future.
  $ cd AMBARI_DIR/ambari-server/ambari/target/ambari-server-*\-dist/ambari-server\-*
  $ java -cp etc/ambari-server/conf/:lib/ambari-server/* org.apache.ambari.server.controller.AmbariServer
  Hit http://localhost:8080 from your web browser

Note:
For quick development turnaround even while running ambari-web with ambari-server integration, create a symbolic link under ambari-server target folder and point it to your ambari-web/public folder
$ cd AMBARI_DIR/ambari-server/ambari/target/ambari-server-dist/ambari-server
$ rm -rf web
$ ln -s AMBARI_DIR/ambari-web/public web
$ cd AMBARI_DIR/ambari-web
$ brunch watch
This way, you can simply modify your source files in AMBARI_DIR/ambari-web and changes will be available immediately from localhost:8080.

Setting up Ambari in Eclipse

Tools needed to build Ambari

The following tools are needed to build Ambari from source.

Alternatively, you can easily launch a VM that is preconfigured with all the tools that you need.  See the Pre-Configured Development Environment section in the Quick Start Guide.

• xCode (if using Mac - free download from the apple store)
• JDK 8 (Ambari 2.6 and below can be compiled with JDK 7, from Ambari 2.7, it can be compiled with at least JDK 8)
• Apache Maven 3.3.9 or later
  Tip: In order to persist your changes to the JAVA_HOME environment variable and add Maven to your path, create the following files:
  
  File: ~/.profile

   

  Code Block
  source ~/.bashrc

   

  File: ~/.bashrc

   

  Code Block
  export PATH=/usr/local/apache-maven-3.3.9/bin:$PATH export JAVA_HOME=$(/usr/libexec/java_home) export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true"

  
  

• Python 2.6 (Ambari 2.7 or later require Python 2.7 as minimum supported version)

• Python setuptools:
  for Python 2.6: Download setuptools and run:

  Code Block
  sh setuptools-0.6c11-py2.6.egg

  for Python 2.7:  Download setuptools and run:

  Code Block
  sh setuptools-0.6c11-py2.7.egg

  
  

• rpmbuild (rpm-build package)

• g++ (gcc-c++ package)

Running Unit Tests

• mvn clean test

• Run unit tests in a single module:

  Code Block
  mvn -pl ambari-server test

  
  

• Run only Java tests:

  Code Block
  mvn -pl ambari-server -DskipPythonTests

  
  

• Run only specific Java tests:

  Code Block
  mvn -pl ambari-server -DskipPythonTests -Dtest=AgentHostInfoTest test

  
  

• Run only Python tests:

  Code Block
  mvn -pl ambari-server -DskipSurefireTests test

  
  

• Run only specific Python tests:

  Code Block
  mvn -pl ambari-server -DskipSurefireTests -Dpython.test.mask=TestUtils.py test

  
  

• Run only Checkstyle and RAT checks:

  Code Block
  mvn -pl ambari-server -DskipTests test

  
  

NOTE: Please make sure you have npm in the path before running the unit tests.

Generating Findbugs Report

• mvn clean install

This will generate xml and html report unders target/findbugs. You can also add flags to skip unit tests to generate report faster.

Building Ambari

Note: if you can an error that too many files are open while building, then run: ulimit -n 10000 (for example)

To build Ambari RPMs, run the following.

Note: Replace ${AMBARI_VERSION} with a 4-digit version you want the artifacts to be (e.g., -DnewVersion=1.6.1.1)

Note: If running into errors while compiling the ambari-metrics package due to missing the artifacts of jms, jmxri, jmxtools:

The work around is to manually install the three missing artifacts:

# Update the revision property to the release version
mvn versions:set-property -Dproperty=revision -DnewVersion=2.8.0.0.0 

mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

# Update the revision property to the release version
mvn versions:set-property -Dproperty=revision -DnewVersion=2.8.0.0.0 
 
mvn -B clean install package rpm:rpm -DskipTests -Psuse11 -Dpython.ver="python >= 2.6" -Preplaceurl

# Update the revision property to the release version
mvn versions:set-property -Dproperty=revision -DnewVersion=2.8.0.0.0 
 
mvn -B clean install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

mvn versions:set -DnewVersion=${AMBARI_VERSION}
 
#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM. 
#Hence you also need to set ambari-metrics project version as well.
pushd ambari-metrics
mvn versions:set -DnewVersion=${AMBARI_VERSION}
popd
 
mvn -B clean install package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

mvn versions:set -DnewVersion=${AMBARI_VERSION}
 
#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM. 
#Hence you also need to set ambari-metrics project version as well.
pushd ambari-metrics
mvn versions:set -DnewVersion=${AMBARI_VERSION}
popd
 
mvn -B clean install package rpm:rpm -DskipTests -Psuse11 -Dpython.ver="python >= 2.6" -Preplaceurl

mvn versions:set -DnewVersion=${AMBARI_VERSION}
 
#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM. 
#Hence you also need to set ambari-metrics project version as well.
pushd ambari-metrics
mvn versions:set -DnewVersion=${AMBARI_VERSION}
popd
 
mvn -B clean install package jdeb:jdeb -DskipTests -Dpython.ver="python >= 2.6" -Preplaceurl

Ambari Server will create following packages

• RPM will be created under AMBARI_DIR/ambari-server/target/rpm/ambari-server/RPMS/noarch.
• DEB will be created under AMBARI_DIR/ambari-server/target/

Ambari Agent will create following packages

• RPM will be created under AMBARI_DIR/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64.
• DEB will be created under AMBARI_DIR/ambari-agent/target

Optional parameters:

• -X -e: add these options for more verbose output by Maven.  Useful when debugging Maven issues.
• -DdefaultStackVersion=STACK-VERSION
• Sets the default stack and version to be used for installation (e.g., -DdefaultStackVersion=HDP-1.3.0)
• -DenableExperimental=true
• Enables experimental features to be available via Ambari Web (default is false)
• All views can be packaged in RPM by adding -Dviews parameter
  • mvn -B clean install package rpm:rpm -Dviews -DskipTests
• Specific views can be built by adding --projects parameter to the -Dviews
  • mvn -B clean install package rpm:rpm --projects ambari-web,ambari-project,ambari-views,ambari-admin,contrib/views/files,contrib/views/pig,ambari-server,ambari-agent,ambari-client,ambari-shell -Dviews -DskipTests
    
    

NOTE: Run everything as root below.

Building Ambari Metrics

If you plan on installing the Ambari Metrics service, you will also need to build the Ambari Metrics project. 

cd ambari-metrics
mvn clean package -Dbuild-rpm -DskipTests

For Ubuntu:
cd ambari-metrics
mvn clean package -Dbuild-deb -DskipTests

Note:

The metrics rpms will be found at: ambari-metrics-assembly/target/. These would be need for installing the Ambari Metrics service.

Running the Ambari Server

First, install the Ambari Server RPM.

On RHEL/CentOS:

yum install ambari-server/target/rpm/ambari-server/RPMS/noarch/ambari-server-*.noarch.rpm 

On SUSE/SLES:

zypper install ambari-server/target/rpm/ambari-server/RPMS/noarch/ambari-server-*.noarch.rpm 

On Ubuntu 12:

 
dpkg --install ambari-server/target/ambari-server-*.deb          # Will fail with missing dependencies errors
apt-get update                                                   # Update locations of dependencies
apt-get install -f                                               # Install all failed dependencies
dpkg --install ambari-server/target/ambari-server-*.deb          # Will succeed

Initialize Ambari Server:

ambari-server setup

Start up Ambari Server:

ambari-server start

See Ambari Server log:

tail -f /var/log/ambari-server/ambari-server.log

To access Ambari, go to

http://{ambari-server-hostname}:8080

from your web browser and log in with username admin and password admin.

Install and Start the Ambari Agent Manually on Each Host in the Cluster

Install the Ambari Agent RPM.

On RHEL/CentOS:

yum install ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ambari-agent-*.rpm

SUSE/SLES:

zypper install ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ambari-agent-*.rpm

Ubuntu12:

dpkg --install ambari-agent/target/ambari-agent-*.deb

Edit the location of Ambari Server in /etc/ambari-agent/conf/ambari-agent.ini by editing the hostname line.

Start Ambari Agent:

ambari-agent start

See Ambari Agent log:

tail -f /var/log/ambari-agent/ambari-agent.log

Setting up Ambari in Eclipse

$ mvn clean eclipse:eclipse

...

After doing the above you should be able to import the project via eclipse's import projectEclipse "Import > Maven > Existing Maven Project". Choose the root directory where you cloned the git repository. You should be able to see the following projects on eclipse:>

ambari
|
|- ambari-project
|- ambari-server

...

|- ambari-agent
|- ambari-web

Select the top-level "ambari pom.xml" and click Finish.

Coding Guidelines for Ambari