...

Open Items

Required Changes

• Knox will Identify the nature of all the supported servicesWhich services are purely topology-based?
  
• WEBHCAT
• HBASE
• OOZIE
• WEBHDFS
?
Which services are currently ZooKeeper-based?
• HIVE (HS2ZooKeeperURLManager)
• HBASE (HBaseZooKeeperURLManager)
• Kafka (KafkaZooKeeperURLManager)
• SOLR (SOLRZooKeeperURLManager)
?
Could ZooKeeper support be added for any services which do no currently support it?
For the ZooKeeper-based-HA services, determine if the ZooKeeper details are available from the service's configuration via Ambari.
Can "HA mode" be determined for every service type from the cluster configuration details? Can Knox dynamically identify HA-configured services, and generate the topology accordingly?
Determine how to leverage the cluster discovery data to generate the ZooKeeper HA configuration for the relevant declared topology services.
Knox should parse referenced provider configurations, rather than just blindly copying their contents.
This would will serve multiple purposes:
• It would be will provide a degree of validation (invalid provider configurations would fail parsing)
• It would will allow a finer level of control over what is serialized to the generated topology file:
  • Knox could will have the opportunity to add cluster-specific details into provider configuration in the generated topologies
  • Knox could reference elements of the provider configuration (e.g., HaProvider) to inform the generation of service elements in the generated topologies.

• Investigate moving Move cluster-specific service HA configuration from the HaProvider configuration to the service declaration.
  
  • This just makes more logical sense
  • Moving cluster-specific details out of the provider configuration will make shared provider configurations applicable to more topologies (i.e., more reusable), across clusters.
  • The cluster-specific configuration could be discovered along with the service URL details, rather than having to be hard-coded.
  • This will be treated as override configuration, so that the existing approach will continue to work for backward-compatibility reasons.

Open Items

• Identify the nature of all the supported services
  • Which services are purely topology-based?
    
    • WEBHCAT
    • HBASE
    • OOZIE
    • WEBHDFS
    • ?
      
      
  • Which services are currently ZooKeeper-based?
    • HIVE (HS2ZooKeeperURLManager)
    • HBASE (HBaseZooKeeperURLManager)
    • Kafka (KafkaZooKeeperURLManager)
    • SOLR (SOLRZooKeeperURLManager)
    • ?
      
      
  • Could ZooKeeper support be added for any services which do no currently support it?
    
    
• For the ZooKeeper-based-HA services, determine if the ZooKeeper details are available from the service's configuration via Ambari.
  
  
• Can "HA mode" be determined for every service type from the cluster configuration details? Can Knox dynamically identify HA-configured services, and generate the topology accordingly?
  
  
• Determine how to leverage the cluster discovery data to generate the ZooKeeper HA configuration for the relevant declared topology services.

Proposed Alternative Provider Configuration File Formats

...