Page History

...

We will need to make sure that we continue to meet the needs that this existing mechanism provides.

Anchor
simpledesc simpledesc

2. Simplified Topology Descriptor

In order to simplify what management UI's need to manage, we should support a much simpler descriptor that contains what is needed to generate an actual topology file at deployment time (see item #3 from the Motivation section).

...

Given that we will have a Service Discovery service that can integrate with Ambari as well as other sources of needed metadata, we should be able to start with merely service and UI names and a reference to a named provider configuration. a simplified topology descriptor.
Once the deployment machinery finds the notices this descriptor, it can pull in the referenced provider configuration, iterate over each of the services, UIs, applications and lookup the details for each.
With the provider configuration and service details we can then generate a fully baked topology.

...

 <service>
   <role>NAMENODE</role>
   <url>hdfs://localhost:8020</url>
 </service>
 <service>
   <role>JOBTRACKER</role>
   <url>rpc://localhost:8050</url>
 </service>
 <service>
   <role>WEBHDFS</role>
   <url>http://c6401.ambari.apache.org:50070/webhdfs</url>
 </service>
 <service>
   <role>WEBHCAT</role>
   <url>http://c6401.ambari.apache.org:50111/templeton</url>
 </service>
 <service>
   <role>OOZIE</role>
   <url>http://localhost:11000/oozie</url>
 </service>
 <service>
   <role>WEBHBASE</role>
   <url>http://localhost:60080</url>
 </service>
 <service>
   <role>HIVE</role>
   <url>http://localhost:10001/cliservice</url>
 </service>
 <service>
   <role>RESOURCEMANAGER</role>
   <url>http://localhost:8088/ws</url>
 </service>
 <service>
   <role>AMBARI</role>
   <url>http://c6401.ambari.apache.org:8080</url>
 </service>

 <ui>
   <role>AMBARIUI</role>
   <url>http://c6401.ambari.apache.org:8080</url>
 </ui>

...

We should also consider how we will discover simple descriptors and I think that we may want to have multiple ways.

Just as we currently do for topology files, we can monitor a directory for new or changed descriptors, and trigger topology generation and deployment upon such events.
This is great for development and small cluster deployments, but for production and larger deployments, we need to be able to accommodate multiple instances of Knox better.
My proposal is that for such deployments we also implement cases is a Zookeeper-based discovery mechanism. Then all Knox instances will pick up the change changes from ZK as the central source of truth, and
perform the necessary generation and deployment of the corresponding topology.

4. Service Discovery

In order to transform the simple descriptor described in #2 into a full topology, we need some source for the relevant details and metadata.

...