Ranger Installation Guide

Summary

This document details the steps involved in installing and configuring Apache Ranger. 

These instructions are for installing Ranger on CentOS/RHEL (release 6).

Prerequisites

• JDK 7+ needs to be installed. Both OpenJDK and Oracle JDK are supported.

• MySQL (5.6+) or ORACLE DB (11g+) for Policy/Audit DB. <TBD: Link on database settings for Ranger>
  • DB server can be installed on the same host. Or Ranger services need to have access to DB server host
  • For production, ensure appropriate capacity planning is done for the database size
• Maven. If not installed, please follow below steps

  • wget http://mirrors.gigenet.com/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz (Use different mirror if needed)

  • su -c "tar -zxvf apache-maven-3.0.5-bin.tar.gz -C /opt/"

  • su -c "vi /etc/profile.d/maven.sh" # Add the following lines to maven.sh 

    • export M2_HOME=/opt/apache-maven-3.0.5 

    • export M2=$M2_HOME/bin 

    • export PATH=$M2:$PATH

  • Now test your install of Maven. Logout of the system and then log back into it. Enter the following command:

  • mvn -version
• Ranger Admin process requires approximately 1.5GB of RAM
  
  

Building Ranger from source

1. Get the ranger source 

• mkdir ~/dev

• cd ~/dev

• git clone -b ranger-0.4  git@github.com:apache/incubator-ranger.git ranger

2. Now build the source

• cd ranger

• export MAVEN_OPTS="-Xmx512M"

• yum search gcc and Install gcc (yum install gcc.x86_64)

• mvn clean compile package assembly:assembly

3. Verify all the tar files under target dir

   ls -ltr *tar.gz
   -rw-r--r-- 1 root root 15068844 Dec 1 04:30 ranger-0.4.0-hdfs-plugin.tar.gz
   -rw-r--r-- 1 root root 14480716 Dec 1 04:30 ranger-0.4.0-hive-plugin.tar.gz
   -rw-r--r-- 1 root root 14349626 Dec 1 04:30 ranger-0.4.0-hbase-plugin.tar.gz
   -rw-r--r-- 1 root root 17763192 Dec 1 04:30 ranger-0.4.0-knox-plugin.tar.gz
   -rw-r--r-- 1 root root 21243470 Dec 1 04:31 ranger-0.4.0-storm-plugin.tar.gz
   -rw-r--r-- 1 root root 126143540 Dec 1 04:31 ranger-0.4.0-admin.tar.gz
   -rw-r--r-- 1 root root 7677999 Dec 1 04:31 ranger-0.4.0-usersync.tar.gz

Install/Configure Ranger Admin

What is done here?

• Ranger Admin bits are laid out

• Database and database users for Ranger Admin policy store and Ranger Audit database are created.

• Ranger Admin properties are configured

• Ranger Admin startup service files are created and updated

Steps

1. Lay down the binaries into appropriate places.  

• cd /usr/local

• sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-admin.tar.gz

• sudo ln -s ranger-0.4.0-admin ranger-admin

2. cd ranger-admin and open install.properties using your text editor (e.g. vi install.properties)

3. Verify the root password that you had picked while installing mysql.  I had chosen root so the relevant section in my install.properties file looks as follows 

   • db_root_user=root
   • db_root_password=root

   • db_host=localhost

4. The install process would create a couple of users in the database for storing administration and audit information, pick passwords for those too.  With my choices here’s how the relevant sections in the install.properties file look now.Ranger allows you to get different authentication modes but for now let’s just leave rest of the things in install.properties file as they are. 

   • # DB information Ranger Policy Store schema
     #
   • db_name=ranger
   • db_user=rangeradmin
   • db_password=rangeradmin
   • # DB UserId for storing auditlog infromation
   • #
   • audit_db_name=ranger (Note the database details used here. This is needed while configuring plugins in later steps)
   • audit_db_user=rangerlogger  
   • audit_db_password=rangerlogger

5. Once all the required properties are updated, execute the below scripts to install ranger admin service.

       Execute : ./setup.sh (This will configure the properties)
       Execute : ./set_globals.sh (This will create the soft links and other folders for logs, etc)

6. Create a valid symlink in /usr/bin/ for start/stop of ranger admin
       cd /usr/bin
       ln -sf /usr/local/ranger-admin/ews/start-ranger-admin.sh ranger-admin-start
       ln -sf /usr/local/ranger-admin/ews/stop-ranger-admin.sh ranger-admin-stop

7. Update ranger-admin service file to link to the start and stop scripts
       vim /etc/init.d/ranger-admin ( Update the Start and Stop commands to point to the created symlinks )

8. Start the Ranger Admin
        service ranger-admin start

9. You can verify by visiting the external URL of the server using browser, for example :
       http://<Host Address>:6080/
       Defaut user and password is admin/admin. Please change it post update.

10. Logs are in ews/logs folder. The path is relative to where you have installed ranger-admin. Check xa_portal.log and catalina.out files for ERROR and WARN log messages
    

Next Steps

• Ranger Admin authentication can be configured to use LDAP or Linux system. Consider configuring one of them in production environment. TBD: Provide link to configure LDAP or Linux for authentication

• Review database capacity for Audit database. It can grow dramatically in HBase or high volume environment. TBD: Provide link DB capacity planning
  

Install/Configure Ranger User Sync

What is done here?

• Ranger UserSync bits are laid out

• Configure Ranger UserSync to synchronize with LDAP/AD or Unix system.

• Configure Ranger UserSync to be the authentication server for Linux users (optional)

• Other Ranger UserSync properties are configured

• Ranger UserSync startup service files are created and updated

Steps

1. Start by extracting out binaries at the appropriate place. 

       cd /usr/local

       sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-usersync.tar.gz

       sudo ln -s ranger-0.4.0-usersync ranger-usersync

       sudo mkdir -p /var/log/ranger-usersync

       sudo chown ranger /var/log/ranger-usersync; sudo chgrp ranger /var/log/ranger-usersync

       cd ranger-usersync

2. Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

        POLICY_MGR_URL=http://localhost:6080 (This should point to the server where Ranger Admin has been installed)

        SYNC_SOURCE=unix (This assumes that the user from the linux/unix system will be synchronized). If you want to use LDAP users, you can find here how to Configure Ranger UserSync for LDAP 

        logdir=/var/log/ranger/usersync

3. Now install the usersync by running the setup command

        export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64 (Change this to your Java install folder)

        ./setup.sh

4. create a valid symlink in /usr/bin/ for start/stop of ranger usersync
        cd /usr/bin
        ln -sf /usr/local/ranger-usersync/start.sh ranger-usersync-start
        ln -sf /usr/local/ranger-admin/ews/stop.sh ranger-usersync-stop

5. update ranger-usersync service file to link to the start and stop scripts 

        vim /etc/init.d/ranger-usersync ( Update the Start and Stop commands to point to the created symlinks )

6. Start the Ranger Usersync
         service ranger-usersync start

7. You can verify by looking at the users tab in Ranger Admin. Unix host users should be sync'ed to ranger.

8. Logs are in logs folder. It is relative to the location where ranger-usersync was installed. Look for usersync.log file for User Sync related errors and auth.log for remote login errors.

Next Steps

• If you are using LDAP or AD, then you will have to configure using to synchronize with LDAP/AD

Install/Configure Ranger HDFS Plugin

Ranger HDFS plugin helps to centralize HDFS authorization policies. To verify that, first Apache Hadoop needs to be installed. If Hadoop is not already installed, follow below steps.

• These instructions were written for Hadoop 2.5.2. So grab that tar (hadoop-2.5.2.tar.gz) and checksum file (hadoop-2.5.2.tar.gz.md5).

• Follow the instructions available on the hadoop site http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html.

• Follow steps given in pseudo distributed mode.

• Make note of the location where you installed hadoop. Here I assume that you have installed it in /usr/local/hadoop.

• Create a user under which we could install and ultimately run the various hadoop processes. And login as that user.

Install/Configure Ranger Hive Plugin

Ranger Hive plugin integrates with Hive to enforce authorization policies. To verify that, first Apache Hive needs to be installed. If not already installed, follow below steps. Version 0.14 is required.

• Download and install Hive by following the instructions available on the apache hive website. Hive Install instructions

          cd /usr/local

          sudo tar zxf ~/dev/apache-hive-0.14.0-bin.tar.gz

          sudo ln -s ~/dev/apache-hive-0.14.0-bin.tar.gz hive

          cd hive

• Note: Hive Server 2 doesn’t start unless HADOOP_VERSION is exported to bashrc 

Now let's follow the below steps to install/configure Ranger Hive plugin.

• Create a repository in Ranger Policy Manager. E.g. "hivedev". The same name needs to be configured during plugin setup

• Extract binaries at the appropriate place.

            cd /usr/local

            sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-hive-plugin.tar.gz

            sudo ln -s ranger-0.4.0-hive-plugin ranger-hive-plugin

            cd ranger-hive-plugin

• Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

            POLICY_MGR_URL=http://localhost:6080

            REPOSITORY_NAME=hivedev

            XAAUDIT.DB.HOSTNAME=localhost

            XAAUDIT.DB.DATABASE_NAME=ranger

            XAAUDIT.DB.USER_NAME=rangerlogger

            XAAUDIT.DB.PASSWORD=rangerlogger

• Now enable the hive-plugin by running the enable-hive-plugin.sh command (Remember to set JAVA_HOME)

            cd /usr/local/ranger-hive-plugin

            ./enable-hive-plugin.sh

• Now edit the xasecure-audit.xml file.Once these changes are done Restart hive. This should start the association of ranger-hive-plugin with hive.

  • cd /usr/local/hive/conf
  • Change the xasecure-audit.xml file. Make sure the JDBC properties are correct.

                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                   <value>jdbc:mysql://localhost/ranger</value>
                   </property>
                   <property>
                   <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                   <value>rangerlogger</value>
                   </property>
                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                   <value>rangerlogger</value>
                   </property>

• You can verify by logging into the Ranger Admin Web interface -> Audit Tab -> Agents

Install/Configure Ranger Knox Plugin

Now let's follow the below steps to install/configure Ranger Knox plugin.

• Create a repository in Ranger Policy Manager. E.g. "knoxdev". The same name needs to be configured during plugin setup
  • Set the URL as https://knox_host:8443/gateway/admin/api/v1/topologies (Replace the knox_host with your fully qualified host name)
• Extract binaries at the appropriate place.
            cd /usr/local
            sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-knox-plugin.tar.gz
            sudo ln -s ranger-0.4.0-knox-plugin ranger-knox-plugin
            cd ranger-knox-plugin
• Now let’s edit the install.properties file. Here are the relevant lines that you should edit:
            POLICY_MGR_URL=http://localhost:6080
            REPOSITORY_NAME=knoxdev
            KNOX_HOME=/usr/local/knox
            XAAUDIT.DB.HOSTNAME=localhost
            XAAUDIT.DB.DATABASE_NAME=ranger
            XAAUDIT.DB.USER_NAME=rangerlogger
            XAAUDIT.DB.PASSWORD=rangerlogger
• Now enable the knox-plugin by running the enable-knox-plugin.sh command (Remember to set JAVA_HOME)
            cd /usr/local/ranger-knox-plugin
            ./enable-knox-plugin.sh
• Now edit the xasecure-audit.xml file.
  • cd /usr/local/knox/conf
  • Put in /usr/<hadoop-installs>/<your version>/knox/ext/ a symbolic link to mysql-connector-java.jar
  • Change the xasecure-audit.xml file. Make sure the JDBC properties are correct.

                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                   <value>jdbc:mysql://localhost/ranger</value>
                   </property>
                   <property>
                   <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                   <value>rangerlogger</value>
                   </property>
                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                   <value>rangerlogger</value>
                   </property>

   

• Once these changes are done Restart Knox (Gateway/LDAP). This should start the association of ranger-knox-plugin with Knox.
• You can verify by logging into the Ranger Admin Web interface -> Audit Tab -> Agents

Install/Configure Ranger Storm Plugin

Ranger Storm plugin integrates with Storm to enforce authorization policies. To verify that, first Apache Storm needs to be installed. If not already installed, follow below steps.

Let’s download and install apache storm from Apache Mirrors.  

• cd /usr/local

• sudo tar zxf ~/dev/apache-storm-0.9.3.tar.gz

• sudo ln -s apache-storm-0.9.3 storm

• cd storm

• Following the instructions available on the apache storm site itself (To install Apache Storm).

Now let's follow the below steps to install/configure Ranger Storm Plugin.

• Create a repository in Ranger Policy Manager. E.g. "stormdev". The same name needs to be configured during plugin setup
  

• cd /usr/local

• sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-storm-plugin.tar.gz

• sudo ln -s ranger-0.4.0-storm-plugin ranger-storm-plugin

• cd ranger-storm-plugin

Now let’s edit the install.properties file.  Here are the relevant lines that you should edit:

• Change the insall.properties file

• POLICY_MGR_URL=http://localhost:6080

• REPOSITORY_NAME=stormdev

• KNOX_HOME=/usr/local/storm

• XAAUDIT.DB.IS_ENABLED=true

• XAAUDIT.DB.HOSTNAME=localhost

• XAAUDIT.DB.DATABASE_NAME=ranger

• XAAUDIT.DB.USER_NAME=rangerlogger

• XAAUDIT.DB.PASSWORD=rangerlogger

 Now enable the storm-plugin by running the enable-storm-plugin.sh command (Remember to set JAVA_HOME)

• cd /usr/local/ranger-storm-plugin

• ./enable-storm-plugin.sh

Now edit the xasecure-audit.xml file. Here are the relevant lines that you should edit :

• cd /usr/local/storm/conf

• change the xasecure-audit.xml file.

<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
<value>jdbc:mysql://localhost/ranger</value>
</property>
<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
<value>rangerlogger</value>
</property>
<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
<value>rangerlogger</value>
</property>

• Once these changes are done Restart Storm

• This should start the association of ranger-storm-plugin  with storm.

• You can verify by logging into the Ranger Admin Web interface ->  Audit -> Agents