THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Page properties | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
Apache Curator is used in order to perform interactions with ZooKeeper in HA mode for Flink. Current set up misses several configurations options, which could be useful in certain Flink deployments.
We want to check all ensure that related available options in Apache Curator and enable are configurable for Flink users. Thus Flink users to configure the way can have all mechanisms to allow Flink interacts with ZooKeeper. Given list of features could be critical for Flink adoption with ZooKeeper in cloud environment. For example, currently is not possible to use ZooKeeper with authorization mechanism along with Flink.
Public Interfaces
There are some new configurations should be exposed for high-availability.zookeeper configuration.
Proposed option | ||
Configuration type | Motivation | |
Ability to fully utilise given set up of ZooKeeper for environment. For example: In certain cases ZooKeeper requires additional | ||
Authorization information. For example list of valid names for ensemble in order to prevent the accidental connecting to a wrong ensemble. | ||
Ability that would enable the user to adjust to different network speeds. | ||
Additional checking for Session expiration above what is provided by ZooKeeper. |
The rest of the options provided by Curator framework are considered as non-useful:
...
An issue arises due to a type mismatch between the Flink configuration parameter high-availability.zookeeper.client.authorization and the corresponding Curator method call. The Curator method anticipates an array of AuthInfo (see method javadoc with signature authorization#List<AuthInfo>) while the Flink configuration for ConfigOptions#mapType() provides a different java type - Map<String, String>. To address resolve this, we can handle suggest the conversion between String and byte[] by executing the getBytes() methodfollowing conversion: Each entry of type Map.Entry<String, String> will be transformed into an AuthInfo object with the constructor AuthInfo(String, byte[]). The field entry.key() will serve as the String scheme value, while the field entry.getValue() will be initially converted to a byte[] using the String#getBytes() method. Subsequently, this byte array will be utilized as byte[] auth parameter during the creation of the AuthInfo.
Compatibility, Deprecation, and Migration Plan
...
We could think about utilising the namespaces. The FLIP could propose adding namespace support for Apache Curator . E.g. metric high-availability.zookeeper.client.<config_option> could be translated into the appropriate <config_option> of the Curator configuration. That would allow to load any parameter supported by these systems.
...