...

• Knox will parse referenced provider configurations, rather than just blindly copying their contents.
  This will serve multiple purposes:
  • It will provide a degree of validation (invalid provider configurations would fail parsing)
  • It will allow a finer level of control over what is serialized to the generated topology file:
    • Knox will have the opportunity to add cluster-specific details into provider configuration in the generated topologies
    • Knox could add/modify the HaProvider config based on cluster details (e.g., HA enabled for a particular service)
    • Knox could reference elements of the provider configuration (e.g., HaProvider) to inform the generation of service elements in the generated topologies.

• Move cluster-specific service HA configuration from the HaProvider configuration to the service declaration.
  
  • This just makes more logical sense
  • Moving cluster-specific details out of the provider configuration will make shared provider configurations applicable to more topologies (i.e., more reusable), across clusters.
  • The cluster-specific configuration could be discovered along with the service URL details, rather than having to be hard-coded in a provider configuration.
  • This will be treated as override configuration, so that the existing approach (i.e., complete configuration as HaProvider param values) will continue to work for , to satisfy backward-compatibility reasonsrequirements.

Open Items

• Name the service-level override param for the service-specific HaProvider param component named 'enabled'
  • 'enabled'
  • 'haEnabled'
  • 'ha.enabled'
    
    
• Identify the nature of all the supported services
  • Which services are currently purely topology-based (DefaultURLManager)?
    
    • WEBHCAT
    • OOZIE
    • WEBHDFS
    • ?
      
      
  • Which services are currently ZooKeeper-based (BaseZooKeeperURLManager)?
    • HIVE (HS2ZooKeeperURLManager)
    • HBASE (HBaseZooKeeperURLManager)
    • Kafka (KafkaZooKeeperURLManager)
    • SOLR (SOLRZooKeeperURLManager)
      
      
  • Could ZooKeeper support be added for any services which do no currently support it?
    • These have zookeeper-related configuration (see table):
      • WEBHDFS
      • OOZIE
      • YARN
      • WEBHCAT
        
        
        
• For the ZooKeeper-based-HA services, determine if the ZooKeeper details are available from the service's configuration via Ambari.
  
  
• Can "HA mode" be determined for every service type from the cluster configuration details? Can Knox dynamically identify HA-configured services, and generate the topology accordingly?
  
  
• Determine how to leverage the cluster discovery data to generate the ZooKeeper HA configuration for the relevant declared topology services.

...