Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3
Note: The following steps are for development purposes only. Ambari 1.7.0 and above exposes the ability to enable ResourceManager High Availability directly from the web client and should be used for real-world use cases.

Prerequisites

  • HDP 2.0+
  • YARN Installed
  • At least 2 hosts in the cluster where YARN is not present on 1 host. These steps assume there are 3 hosts in the cluster.

...

  1. The yarn-site configuration group (yarn-site.xml) must now be updated to reflect the backup ResourceManager. 

    propertyvalue
    yarn.resourcemanager.ha.enabledtrue
    yarn.resourcemanager.ha.rm-idsrm1,rm2
    yarn.resourcemanager.hostname.rm1<rm-host-active>
    yarn.resourcemanager.hostname.rm2<rm-host-standby>
    yarn.resourcemanager.recovery.enabledtrue
    yarn.resourcemanager.store.classorg.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
    yarn.resourcemanager.zk-address<zk-host-1>, <zk-host-2>, <zk-host-3>
    yarn.resourcemanager.cluster-id<yarn-cluster-name>
    yarn.resourcemanager.ha.automatic-failover.zk-base-path/yarn-leader-election

     

     

  2. There are two methods to update these configuration properties:
    1. /var/lib/ambari-server/resources/scripts/configs.sh
    2. REST API

  3. configs.sh

    Code Block
    $ ./configs.sh set <ambari-server> <cluster-name> yarn-site <param-name> <param-value>
     
    example
    $ ./configs.sh set c6401.ambari.apache.org cl1 yarn-site yarn.resourcemanager.ha.enabled true

    This will need to be execute for each parameter listed.

     

  4. REST API

    1. Get the existing yarn-site configuration version and tag.

      Code Block
      GET http://<ambari-server>/api/v1/clusters/<cluster-name>?fields=Clusters/desired_configs
      ...
      {
        "href" : "http://<ambari-server>/api/v1/clusters/<cluster-name>?fields=Clusters/desired_configs",
        "Clusters" : {
          "cluster_name" : "c1",
          "version" : "HDP-2.1",
          "desired_configs" : {
            ...
            "yarn-site" : {
              "tag" : "version1",
              "user" : "_anonymous",
              "version" : 1
            },
            ...
          }
        }
      }
    2. Using the value from the tag property of yarn-site, request the current YARN configuration. In this example, <yarn-site-tag> would be "version1".

      Code Block
      GET http://<ambari-server>/api/v1/clusters/<cluster-name>/configurations?(type=yarn-site&tag=<yarn-site-tag>)
      ...
        "href" : "http://<ambari-server>/api/v1/clusters/<cluster-name>/configurations?(type=yarn-site&tag=version1)",
        "items" : [
          {
            "href" : "http://<ambari-server>/api/v1/clusters/<cluster-name>/configurations?type=yarn-site&tag=version1",
            "tag" : "version1",
            "type" : "yarn-site",
            "version" : 1,
            "Config" : {
              "cluster_name" : "c1"
            },
            "properties" : {
              "yarn.acl.enable" : "false",
              "yarn.admin.acl" : "",
              ...
              "yarn.timeline-service.ttl-enable" : "true",
              "yarn.timeline-service.ttl-ms" : "2678400000",
              "yarn.timeline-service.webapp.address" : "c6402.ambari.apache.org:8188",
              "yarn.timeline-service.webapp.https.address" : "c6402.ambari.apache.org:8190"
            }
          }
        ]
      }
    3. The properties property contains the current yarn-site configuration. Append the new properties to this structure and then update yarn-site.

      Code Block
      PUT http://<ambari-server>/api/v1/clusters/<cluster-name>
      
      {  
         "Clusters":{  
            "desired_config":{  
               "type":"yarn-site",
               "tag":"<new-yarn-site-tag>",
               "properties"::{  
                  "yarn.acl.enable":"false",
                  "yarn.admin.acl":"",
                  ... 
                  "yarn.timeline-service.ttl-enable":"true",
                  "yarn.timeline-service.ttl-ms":"2678400000",
                  "yarn.timeline-service.webapp.address":"c6402.ambari.apache.org:8188",
                  "yarn.timeline-service.webapp.https.address":"c6402.ambari.apache.org:8190"
                  ...
                  <yarn-ha-site-properties>
                  "yarn.resourcemanager.ha.enabled": "true",
                  "yarn.resourcemanager.ha.rm-ids": "rm1,rm2",
      			...
               }
            }
         }
      }
  5. Restart all stopped services.