THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Service Config | Property | Example Value |
|---|---|---|
| hive-site | hive.zookeeper.quorum | c6801.ambari.apache.org:2181,c6802.ambari.apache.org:2181,c6803.ambari.apache.org:2181 |
| hive.server2.zookeeper.namespace | hiveserver2 | |
| hive.server2.support.dynamic.service.discovery | true | |
| hdfs-site | ha.zookeeper.quorum | c6801.ambari.apache.org:2181,c6802.ambari.apache.org:2181,c6803.ambari.apache.org:2181 |
| dfs.ha.automatic-failover.enabled | true (only for "Auto HA") | |
| hbase-site | hbase.zookeeper.quorum | c6801.ambari.apache.org:2181,c6802.ambari.apache.org:2181,c6803.ambari.apache.org:2181 |
| zookeeper.znode.parent | /hbase-unsecure | |
| kafka-broker | sandbox.hortonworks.com:2181 | |
| oozie-site | oozie.zookeeper.connection.string | localhost:2181 |
| oozie.zookeeper.namespace | oozie | |
| yarn-site | | true |
| c6801.ambari.apache.org:2181,c6802.ambari.apache.org:2181,c6803.ambari.apache.org:2181 | |
| webhcat-site | templeton.zookeeper.hosts | c6801.ambari.apache.org:2181,c6802.ambari.apache.org:2181,c6803.ambari.apache.org:2181 |
Required Changes
- Knox will parse referenced provider configurations, rather than just blindly copying their contents.
This will serve multiple purposes:- It will provide a degree of validation (invalid provider configurations would fail parsing)
- It will allow a finer level of control over what is serialized to the generated topology file:
- Knox will have the opportunity to add cluster-specific details into provider configuration in the generated topologies
- Knox could reference elements of the provider configuration (e.g., HaProvider) to inform the generation of service elements in the generated topologies.
...
- Name the service-level override param for the service-specific HaProvider param component named 'enabled'
- 'enabled'
- 'haEnabled'
- 'ha.enabled'
- Identify the nature of all the supported services
- Which services are purely topology-based (DefaultURLManager)?
- WEBHCAT
- HBASE
- OOZIE
- WEBHDFS
- ?
- Which services are currently ZooKeeper-based (BaseZooKeeperURLManager)?
- HIVE (HS2ZooKeeperURLManager)
- HBASE (HBaseZooKeeperURLManager)
- Kafka (KafkaZooKeeperURLManager)
- SOLR (SOLRZooKeeperURLManager)
- ? WEBHDFS (supported by the service, but not implemented in Knox) ?
- ? OOZIE (supported by the service, but not implemented in Knox) ?
- ? YARN (supported by the service, but not implemented in Knox) ?
- ? WEBHCAT (supported by the service, but not implemented in Knox) ?
- Could ZooKeeper support be added for any services which do no currently support it?
- Which services are purely topology-based (DefaultURLManager)?
- For the ZooKeeper-based-HA services, determine if the ZooKeeper details are available from the service's configuration via Ambari.
- Can "HA mode" be determined for every service type from the cluster configuration details? Can Knox dynamically identify HA-configured services, and generate the topology accordingly?
- Determine how to leverage the cluster discovery data to generate the ZooKeeper HA configuration for the relevant declared topology services.
...