Page History

...

We should also consider how we will discover simple descriptors and I think that we may want to have multiple ways.

3.1.1 Local

Just as we is currently do done for topology files, we Knox can monitor a local directory for new or changed descriptors, and trigger topology generation and deployment upon such events.
This is great for development and small cluster deployments.

3.1.2 Remote

For , but for production and larger deployments, we need to be able to accommodate multiple instances of Knox better.
My proposal for such cases is a ZooKeeper-based discovery mechanism.
Then, all Knox instances will pick up the changes from ZK as the central source of truth, and
perform the necessary generation and deployment of the corresponding topology.

...

In order to transform the Simplified Topology Descriptor into a full topology, we need some source for the relevant details and metadata. Knox is often deployed in clusters
with Ambari and with ZooKeeper also deployed, and both contain some level of the needed metadata about the service URLs wtihin the cluster.The ZooKeeper based service registry has promise but we need to invetigate the level of use from a registered services perspective and details available for each service.

Ambari should contain provides everything we need to flesh out a full topology from a minimal description of what resources that we want exposed and how we want access to those resources protected.

The ZooKeeper service registry has promise, but we need to investigate the level of use from a registered services perspective and details available for each service.

Anchor
ambariapi ambariapi

4.1 Apache Ambari API

These are some excerpts of responses from the Ambari REST API which can be employed for topology discovery:

...

Since the service URLs for a cluster are being will be discovered, Knox has the opportunity to respond dynamically to subsequent topology changes. For a Knox topology that has been generated and deployed, it's possible that the URL for a given service could subsequently change at some point afterward.

The host name could change. The scheme and/or port could change (e.g., http --> https). The potential and frequency of such changes certainly varies among deployments.

We should consider providing the option for Knox to detect topology changes for a cluster, and respond by updating its corresponding topology.

For example, Ambari provides the ability to request the active configuration versions for all the service components in a cluster. There could be a thread that checks this set, notices one or more version changes, and initiates the re-generates/deploys the generation/deployment of that topology.

5. Provider Configurations

...