Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Status

Current stateIn-progressComplete

Discussion thread

JIRAKNOX-1006

...

  • Service discovery type
    An identifier indicating which type of discovery to apply (e.g., Ambari, etc...)
  • Service discovery address
    The associated service registry address
  • Credentials for interacting with the discovery source
  • A provider configuration reference (a unique name, filename, etc...)
    A unique name mapped to a set of provider configurations  (see item #3 from the Motivation section)
  • A list of services to be exposed through Knox (with optional service parameters and URL values)
  • A list of UIs to be proxied by Knox (per KIP-9)

...

Code Block
languagetext
titleProposed YAML
# Discovery info source
discovery-type: AMBARI
discovery-address: http://c6401sandbox.ambarihortonworks.apache.orgcom:8080
discovery-user: ambariusermaria_dev
discovery-pwd-alias: ambari.discovery.password

# Provider config reference, the contents of which will be
# included in (or referenced from) the resulting topology descriptor.
# The contents of this reference has a <gateway/> root, and
# contains <provider/> configurations.
provider-config-ref : ambarisandbox-cluster-policyproviders.xml

# The cluster for which the service details should be discovered
cluster: myclusterSandbox

# The services to declare in the resulting topology descriptor,
# whose URLs will be discovered (unless a value is specified)
services:
    - name: NAMENODE
    - name: JOBTRACKER
    - name: WEBHDFS
    - name: WEBHCAT
    - name: OOZIE
    - name: WEBHBASE
    - name: HIVE
    - name: RESOURCEMANAGER
    - name: AMBARIKNOXSSO
      urlsparams:
        - http://c6401.ambari.apache.org:8080  knoxsso.cookie.secure.only: true
          knoxsso.token.ttl: 100000 
    - name: AMBARIUIAMBARI
      urls:
        - http://c6401sandbox.ambarihortonworks.apache.orgcom:8080

#  UIs to be- proxiedname: throughAMBARIUI
 the resulting Knox topology   urls:
        - http://sandbox.hortonworks.com:8080

# UIs to be proxied through the resulting Knox topology (see KIP-9)
#uis:
#   - name: AMBARIUI
#     url: http://c6401sandbox.ambarihortonworks.apache.orgcom:8080

 

While JSON is not really a format for configuration, it is certainly appropriate as a wire format, and will be used for API interactions.

Code Block
languagetext
titleProposed JSON
{
  "discovery-type":"AMBARI",
  "discovery-address":"http://c6401sandbox.ambarihortonworks.apache.orgcom:8080",
  "discovery-user":"ambariusermaria_dev",
  "discovery-pwd-alias":"ambari.discovery.password",
  "provider-config-ref":"ambarisandbox-cluster-policyproviders.xml",
  "cluster":"myclusterSandbox",
  "services":[
     {"name":"NAMENODE"},
     {"name":"JOBTRACKER"},
     {"name":"WEBHDFS"},
     {"name":"WEBHCAT"},
     {"name":"OOZIE"},
     {"name":"WEBHBASE"},
     {"name":"HIVE"},
     {"name":"RESOURCEMANAGER"},
     {"name":"AMBARIKNOXSSO",
      "urls":["http://c6401.ambari.apache.orgparams":{
          "knoxsso.cookie.secure.only":"true",
          "knoxsso.token.ttl":"100000"
      }
     },
     {"name":"AMBARI", "urls":["http://sandbox.hortonworks.com:8080"]}
  ],
  "uis":[
     {"name":"AMBARIUI", "urls":["http://c6401sandbox.ambarihortonworks.apache.orgcom:8080"]}
  ]
} 

 

Anchor
topologygen
topologygen
3.Topology Generation

...

Code Block
languagexml
titleSample Topology File
<?xml version="1.0" encoding="UTF-8"?>
<topology>
    <gateway>
        <provider>
            <role>authentication</role>
            <name>ShiroProvider</name>
            <enabled>true</enabled>
            <param>
                <!--
                session timeout in minutes,  this is really idle timeout,
                defaults to 30mins, if the property value is not defined,,
                current client authentication would expire if client idles contiuosly for more than this value
                -->
                <name>sessionTimeout</name>
                <value>30</value>
            </param>
            <param>
                <name>main.ldapRealm</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
            </param>
            <param>
                <name>main.ldapContextFactory</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
            </param>
            <param>
                <name>main.ldapRealm.contextFactory</name>
                <value>$ldapContextFactory</value>
            </param>
            <param>
                <name>main.ldapRealm.userDnTemplate</name>
                <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org<org</value>
            </param>
            <param>
                <name>main.ldapRealm.contextFactory.url</name>
                <value>ldap://localhost:33389</value>
            </param>
            <param>
                <name>main.ldapRealm.contextFactory.url<authenticationMechanism</name>
                <value>ldap://localhost:33389<<value>simple</value>
            </param>
            <param>
                <name>main.ldapRealm.contextFactory.authenticationMechanism<<name>urls./**</name>
                <value>simple<<value>authcBasic</value>
            </param>
        </provider>
        <provider>
            <param>
<role>identity-assertion</role>
            <name>Default</name>
            <name>urls./**</name><enabled>true</enabled>
        </provider>
        <!--
         <value>authcBasic</value>
            </param>Defines rules for mapping host names internal to a Hadoop cluster to externally accessible host names.
        </provider>
        <provider>
            <role>identity-assertion</role>
For example, a hadoop service running in AWS may return a response that includes URLs containing the
        some AWS internal host name.  If the <name>Default</name>
client needs to make a subsequent request to the host identified
  <enabled>true</enabled>
      in those </provider>
URLs they need to be mapped to  <!--
        Defines rules for mapping host names internal to a Hadoop cluster to externally accessible host names.
        For example, a hadoop service running in AWS may return a response that includes URLs containing the
        some AWS internal host name.  If the client needs to make a subsequent request to the host identified
        in those URLs they need to be mapped to external host names that the client Knox can use to connect.
        If the external hostname and internal host names are same turn of this provider by setting the value of
        enabled parameter as false.
        The name parameter specifies theexternal host names that the client Knox can use to connect.
        If the external hostname and internal host names are same turn of this provider by setting the value of
        enabled parameter as false.
        The name parameter specifies the external host names in a comma separated list.
        The value parameter specifies corresponding internal host names in a comma separated list.
        Note that when you are using Sandbox, the external hostname needs to be localhost, as seen in out
        of box sandbox.xml.  This is because Sandbox uses port mapping to allow clients to connect to the
        Hadoop services using localhost.  In real clusters, external host names inwould aalmost commanever separatedbe listlocalhost.
        The value parameter specifies corresponding internal host names in a comma separated list.-->
        <provider>
        Note that when you are<role>hostmap</role>
 using Sandbox, the external hostname needs to be localhost, as seen in out <name>static</name>
        of box sandbox.xml.  This<enabled>true</enabled>
 is because Sandbox uses port mapping to allow clients to connect to the
 <param><name>localhost</name><value>sandbox,sandbox.hortonworks.com</value></param>
        </provider>
 Hadoop services using localhost. </gateway>
 
  In real clusters,<service>
 external host names would almost never be localhost.<role>AMBARIUI</role>
        -->
    <url>http://c6401.ambari.apache.org:8080</url>
    <provider></service>
    <service>
        <role>hostmap<<role>HIVE</role>
        <url>http://c6402.ambari.apache.org:10001/cliservice</url>
    <name>static<</name>service>
    <service>
        <enabled>true<<role>WEBHCAT</enabled>role>
            <param><name>localhost</name><value>sandbox,sandbox.hortonworks.com</value></param>
    <url>http://c6402.ambari.apache.org:50111/templeton</url>
    </provider>service>
    </gateway>
 
    <service>
        <role>AMBARIUI<<role>AMBARI</role>
        <url>http://c6401.ambari.apache.org:8080</url>
    </service>
    <service>
        <role>HIVE<<role>OOZIE</role>
        <url>http://c6402.ambari.apache.org:1000111000/cliservice<oozie</url>
    </service>
    <service>
        <role>WEBHCAT<<role>JOBTRACKER</role>
        <url>http<url>rpc://c6402.ambari.apache.org:501118050</templeton</url>
    </service>
    <service>
        <role>AMBARI<<role>NAMENODE</role>
        <url>http<url>hdfs://c6401.ambari.apache.org:8080<8020</url>
    </service>
    <service>
        <role>OOZIE<<role>WEBHBASE</role>
        <url>http://c6402c6401.ambari.apache.org:11000/oozie<60080</url>
    </service>
    <service>
        <role>JOBTRACKER<<role>WEBHDFS</role>
        <url>rpc<url>http://c6402c6401.ambari.apache.org:8050<50070/webhdfs</url>
    </service>
    <service>
        <role>NAMENODE<<role>RESOURCEMANAGER</role>
        <url>hdfs<url>http://c6401c6402.ambari.apache.org:8020<8088/ws</url>
    </service>
    <service>
        <role>WEBHBASE<<role>KNOXSSO</role>
        <param>
           <url>http://c6401.ambari.apache.org:60080</url>
 <name>knoxsso.cookie.secure.only</name>
        </service>
    <service><value>true</value>
        <role>WEBHDFS<</role>param>
        <url>http://c6401.ambari.apache.org:50070/webhdfs</url>
 <param>
            </service><name>knoxsso.token.ttl</name>
    <service>
        <role>RESOURCEMANAGER<<value>100000</role>value>
        <url>http://c6402.ambari.apache.org:8088/ws</url></param>
    </service>
</topology>


3.1 Simple Descriptor Discovery

...

  1. Provision the alias mapping using the knoxcli.sh script

    bin/knoxcli.sh create-alias ambari.discovery.user --value ambariuser

  2. Specify the discovery-user property in a descriptor (This can be useful if a Knox instance will proxy services in clusters managed by multiple Ambari instances)

    "discovery-user":"ambariuser"

...

  1. Provision the password mapped to the default alias, ambari.discovery.password

    bin/knoxcli.sh create-alias ambari.discovery.password --value ambaripasswd

  2. Provision a different alias, and specify it in the descriptordescriptor (This can be useful if a Knox instance will proxy services in clusters managed by multiple Ambari instances)

    "discovery-pwd-alias":"my.ambari.discovery.password.alias"

 

Related Links