Versions Compared

Old Version 57

changes.mady.by.user Gautam Borad

Saved on

compared with

New Version Current

changes.mady.by.user Gautam Borad

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Creating Keytab and principals

Note: Below steps required only for manual installation of ranger services and plugins

...

 -> kdestroy (Please don’t miss kdestroy after above step)

 Note: Change the keytab permission to read only and assign it to “ranger” user


Installation Steps for Ranger-Admin

  1. Untar the ranger-<verison>-admin.tar.gz

    -> tar zxf ranger-<version>-admin.tar.gz

  2. Change directory to ranger-<version>-admin

    -> cd ranger-<version>-admin

  3.  Edit install.properties (Enter appropriate values for the below given properties)

 

db_root_user=<username>
db_root_password=<password of db>
db_host=test-dummy-X.openstacklocal
db_name=
db_user=
db_password=
policymgr_external_url=http://<FQDN_OF_Ranger_Admin_Cluster>:6080
authentication_method=UNIX or LDAP or AD
spnego_principal=HTTP/<FQDN_OF_Ranger_Admin_Cluster>@<REALM>
spnego_keytab=<HTTP keytab path>
token_valid=30
cookie_domain=<FQDN_OF_Ranger_Admin_Cluster>
cookie_path=/
admin_principal=rangeradmin/<FQDN_OF_Ranger_Admin_Cluster>@<REALM>
admin_keytab=<rangeradmin keytab path>
lookup_principal=rangerlookup/<FQDN_OF_Ranger_Admin_Cluster>@<REALM>
lookup_keytab=<rangerlookup keytab path>
hadoop_conf=/etc/hadoop/conf

Note: If kerberos server and admin are on different host then copy the keytab on admin host and assign permission to “ranger” user

  • scp the rangeradmin keytab file to the respective path of another host
  • chown ranger <rangeradmin keytab path>
  • chmod 400 <rangeradmin keytab path>

4. Run setup   

    -> ./setup.sh

5. Start Ranger admin server 

    -> ./ranger-admin-services.sh start 

Installation Steps for Ranger-Usersync

  1. Untar the ranger-<verison>-usersync.tar.gz

    -> tar zxf ranger-<version>-usersync.tar.gz

  2. Change directory to ranger-<version>-usersync

    -> cd ranger-<version>-usersync

  3.  Edit install.properties (Enter appropriate values for the below given properties)

 

POLICY_MGR_URL =http://<FQDN_OF_Ranger_Admin_Cluster>:6080
usersync_principal=rangerusersync/test-dummy-X.openstacklocal@<REALM>
usersync_keytab=<rangerusersync keytab path>
hadoop_conf=/etc/hadoop/conf

Note: If kerberos server and usersync are on different host then copy the keytab on usersync host and assign permission to “ranger” user

  • scp the rangerusersync keytab file to the respective path of another host
  • chown ranger <rangeusersync keytab path>
  • chmod 400 <rangerusersync keytab path>

4. Run setup   

   -> ./setup.sh

5. Start Usersync server

   ->  ./ranger-usersync-services.sh start 

Installation Steps for Ranger-Tagsync

  1. Untar the ranger-<verison>-tagsync.tar.gz

    -> tar zxf ranger-<version>-tagsync.tar.gz

  2. Change directory to ranger-<version>-tagsync

    -> cd ranger-<version>-tagsync

  3.  Edit install.properties (Enter appropriate values for the below given properties)

 

 
TAGADMIN_ENDPOINT =http://<FQDN_OF_Ranger_Admin_Cluster>:6080
 tagsync
 tagsync_principal=rangertagsync/test-dummy-X.openstacklocal@<REALM>
 tagsync
 tagsync_keytab=<rangertagsync keytab path>
 hadoop
 hadoop_conf=/etc/hadoop/conf
TAG_SOURCE= (either 'atlas' or 'file' or 'atlasrest')

Note: If kerberos server and tagsync are on different host then copy the keytab on tagsync host and assign permission to “ranger” user

  • scp the rangertagsync keytab file to the respective path of another host
  • chown ranger <rangetagsync keytab path>
  • chmod 400 <rangertagsync keytab path>

4. Run setup   

   -> ./setup.sh

5. Start Ranger tagsync server 

   -> ./ranger-tagsync-services.sh start

Installation Steps for Ranger-KMS

  1. Untar the ranger-<verison>-SNAPSHOT-kms.tar.gz

    -> tar zxf ranger-<version>-SNAPSHOT-kms.tar.gz

  2. Change directory to ranger-<version>-SNAPSHOT-kms

    -> Cd ranger-<version>-SNAPSHOT-kms

  3.  Edit install.properties (Enter appropriate values for the below given properties)

 

  
  1. KMS_MASTER_KEY_PASSWD=<Master Key Password>
 
  1. kms_principal=rangerkms/<FQDN of ranger kms host>@<REALM>
 
  1. kms_keytab=<ranger kms keytab path>
 
  1. hadoop_conf=<hadoop core-site.xml path>
    POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

Note: if kerberos server and Ranger KMS are on different host then copy the keytab on Ranger KMS host and assign permission to “kms” user

  • scp the rangerkms keytab file to the respective path
  • chown ranger <rangekms keytab path>
  • chmod 400 <rangerkms keytab path>

4. Run setup   

   -> ./setup.sh

5. Follow other setup required for kerberized cluster like creating keytab adding proxy user

6. Start Ranger tagsync server 

   -> ./ranger-kms start

Installing Ranger Plugins Manually

Installing/Enabling Ranger HDFS plugin

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-hdfs-plugin.tar.gz to nameNode host in /usr/hdp/<hdp-version> directory 

    -> cd ranger-<version>-SNAPSHOT-hdfs-plugin

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-hdfs-plugin.tar.gz 

    ->cd ranger-<version>-SNAPSHOT-hdfs-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=hadoopdev

    -> Audit info (Solr/HDFS options available)

  4. Enable the HDFS plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

    -> ./enable-hdfs-plugin.sh

  5. After enabling plugin, follow the below steps to stop/start namenode.

    ->su hdfs -c "/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh stop namenode"

    ->su hdfs -c "/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh start namenode"


  6.  Create the default repo for HDFS with proper configuration. 

    -> In Custom repo config add component user (eg. hdfs) as value for below properties

    1.  policy.download.auth.users

    7. OR

    1.  OR policy.grantrevoke.auth.users

    2. tag.download.auth.users


  8. You can verify the plugin is communicating to ranger admin in Audit->plugins tab

    .

Installing/Enabling Ranger HIVE plugin

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-hive-plugin.tar.gz to hiveServer2 host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-hive-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-hive-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=hivedev

    -> Audit info (Solr/HDFS options available)

  4. Enable the Hive plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

    -> ./enable-hive-plugin.sh

  5. After enabling plugin, follow the below steps to stop/start hiveserver2.

    -> ps -aux | grep hive | grep -i hiveserver2 | awk '{print $1,$2}' | grep hive | awk '{print $2}' | xargs kill >/dev/null 2>&1

    -> su hive -c "nohup /usr/hdp/current/hive-server2/bin/hiveserver2 -hiveconf hive.metastore.uris="" -hiveconf hive.log.dir=/var/log/hive -hiveconf hive.log.file=hiveserver2.log >/var/log/hive/hiveserver2.out 2> /var/log/hive/hiveserver2err.log &"

  6.  Create the default repo for Hive with proper configuration

     -> > In Custom repo config add component user (eg. hive) as value for below properties

    7.  

    1. policy.download.auth.users OR policy.grantrevoke.auth.users

    2. tag.download.auth.users


  8. You can verify the plugin is communicating to ranger admin in Audit->plugins tab

    .

 

 Installing/Enabling Ranger HBASE plugin

 

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-hbase-plugin.tar.gz to Active Hbasemaster host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-hbase-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-hbase-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=hbasedev

    -> Audit info (Solr/HDFS options available)

  4. Enable the Hbase plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

    -> ./enable-hbase-plugin.sh

  5. After enabling plugin, follow the below steps to stop/start hbase.

    -> su hbase -c "/usr/hdp/current/hbase-master/bin/hbase-daemon.sh stop regionserver; sleep 25"

    -> su hbase -c "/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh stop master"

    -> su hbase -c "/usr/hdp/current/hbase-master/bin/hbase-daemon.sh start master; sleep 25"

    -> su hbase -c "/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh start regionserver"


  6.  Create the default repo for for HBase with  with proper configurationconfigurations

     -> In Custom repo config add component user (eg. hbase) as value for below properties

    7.  

    1. policy.grantrevoke.auth.users

    2. tag.download.auth.users


  8. You can verify the plugin is communicating to ranger admin in Audit->plugins tab.

Installing/Enabling Ranger YARN plugin

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-yarn-plugin.tar.gz to Active ResourceManager host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-yarn-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-yarn-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=yarndev

    -> Audit info (Solr/HDFS options available)

  4. Enable the YARN plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64

    -> ./enable-yarn-plugin.sh

  5. Make sure HADOOP_YARN_HOME and HADOOP_LIBEXEC_DIR is set.

    -> export HADOOP_YARN_HOME=/usr/hdp/current/hadoop-yarn-nodemanager/

    -> export HADOOP_LIBEXEC_DIR=/usr/hdp/current/hadoop-client/libexec/ 


  6.  After After enabling plugin, follow the below steps to stop/start.

    -> Stop Stop/Start the ResourceManager on all your ResourceManager hosts.

    1. su yarn -c "/usr/hdp/current/hadoop-yarn-resourcemanager/sbin/yarn-daemon.sh stop resourcemanager"

    2. su yarn -c "/usr/hdp/current/hadoop-yarn-resourcemanager/sbin/yarn-daemon.sh start resourcemanager"

      3.  

    3.   

    4. ps -ef | grep -i resourcemanager  

    -

    >

    > Stop/Start the NodeManager on all your NodeManager hosts. 

    1. su yarn -c "/usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh stop nodemanager"
    2. su yarn -c "/usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh start nodemanager"

    3. ps -ef | grep -i nodemanager

    7.  


  8. Create the default repo for Yarn with proper configuration  In Custom repo config add component user (eg. yarn) as value for below propertiesYou can verify the plugin is communicating to ranger admin in Audit->plugins tab. 

    1. policy.download.auth.users
    9. OR
    1.  OR policy.grantrevoke.auth.users
    2. tag.download.auth.users
  10.  You can verify the plugin is communicating to ranger admin in Audit->plugins tab.

Installing/Enabling Ranger KNOX plugin 

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-knox-plugin.tar.gz to Active ResourceManager host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-knox-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-knox-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=knoxdev

    -> Audit info (Solr/HDFS options available)

  4. Enable the Knox plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64

    -> ./enable-knox-plugin.sh

  5. After enabling plugin, follow the below steps to stop/start knox gateway.

    -> su knox -c "/usr/hdp/current/knox-server/bin/gateway.sh stop"

    -> su knox -c "/usr/hdp/current/knox-server/bin/gateway.sh start"


  6.  Create the default repo for Knox with proper configuration

     -> > In Custom repo config add component user (eg. hive) as value for below properties

    7.  

    1. policy.grantrevoke.auth.users

    2. tag.download.auth.users

  8. You

    You can verify the plugin is communicating to ranger admin in Audit->plugins tab.

    Note: 
       -> For Test Connection to be successful follow addition step Trusting Self Signed Knox Certificate
       -> Knox plugin must be enabled in all Knox instances (in HA env).  

Installing/Enabling Ranger STORM plugin

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-storm-plugin.tar.gz to Active host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-storm-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-storm-plugin

  3. Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=stormdev

    -> Audit info (Solr/HDFS options available)

  4. Enable the Storm plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

    -> ./enable-storm-plugin.sh

  5. After enabling plugin, follow the below steps to stop/start storm.

    -> su - storm -c 'source /usr/hdp/current/storm-nimbus/conf/storm-env.sh ; export PATH=/usr/jdk64/jdk1.8.0_60/bin:$PATH ; storm nimbus > /var/log/storm/nimbus.out'

    -> su - storm -c 'source /usr/hdp/current/storm-client/conf/storm-env.sh ; export PATH=/usr/jdk64/jdk1.8.0_60/bin:$PATH ; storm drpc > /var/log/storm/drpc.out'

    -> su - storm -c 'source /usr/hdp/current/storm-client/conf/storm-env.sh ; export PATH=/usr/jdk64/jdk1.8.0_60/bin:$PATH ; storm ui > /var/log/storm/ui.out'

    -> su - storm -c 'source /usr/hdp/current/storm-supervisor/conf/storm-env.sh ; export PATH=/usr/jdk64/jdk1.8.0_60/bin:$PATH ; storm logviewer > /var/log/storm/logviewer.out'


  6.  Create the default repo for Storm with proper configuration

    -> > In Custom repo config add component user (eg. storm) as value for below properties

    7.  

    1. policy.grantrevoke.auth.

    8. users OR

    1. users OR policy.grantrevoke.auth.users

    2. tag.download.auth.users


  9. You can verify the plugin is communicating to ranger admin in Audit->plugins tab.

Installing/Enabling Ranger KAFKA plugin

 

  1. We’ll start by extracting our build at the appropriate place

    -> copy ranger-<version>-SNAPSHOT-kafka-plugin.tar.gz to Active host in /usr/hdp/<hdp-version> directory

    -> cd /usr/hdp/<hdp-version>

  2.  Untar the ranger-<verison>-SNAPSHOT-SNAPSHOT-kafka-plugin.tar.gz

    -> cd ranger-<version>-SNAPSHOT-kafka-plugin

  3.  Edit the install.properties file.  Here are the relevant lines that you should edit:

    -> COMPONENT_INSTALL_DIR_NAME=/usr/hdp/<hdp-version>/kafka

    4.  

  4. -> POLICY_MGR_URL=http://<FQDN of ranger admin host>:6080

    -> REPOSITORY_NAME=kafkadev

    -> Audit info (Solr/HDFS options available)

  5. Enable the KAFKA plugin by running the below commands

    -> export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

    -> ./enable-kafka-plugin.sh
     

  6.  Create the default repo for Storm with proper configuration

    -> In Custom repo config add component user (eg. storm) as value for below properties

    1.  policy.grantrevoke.auth.users OR policy.grantrevoke.auth.users

    2. tag.download.auth.users

  7. You

    You can verify the plugin is communicating to ranger admin in Audit->plugins tab.

Note: 


   ->

If plugin is not able to communicate then check property

“authorizer

“authorizer.class.

name”

name” in /usr/hdp/<hdp-version>/kafka/config/server.properties, value of authorizer.class.name should be org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.