Design

Most of the configuration parameters and files are common for all the instances of the same OODT component. Therefore, following ZNode structure is adapted where configuration related to each individual component type is stored in a separate ZNode sub tree as shown below.

The Idea

On a high level, there can be multiple projects running different sets of file managers, workflow managers and etc. To store configuration for different projects separately in zookeeper, the concept of projects are introduced and the root ZNode is divided to sub trees based on the project. Under each project there are configuration stored for different OODT components (file manager, resource manager, ...). Basically, there are two types of configuration files that needs to be stored, properties files and other configuration files (like XML files and etc). Out of those, properties within the properties files are only loaded once (normally at the initialization). Therefore, they are treated as an special case in contrast to other configuration files. In order to achieve this, two sub trees are created as shown above for properties files and other configuration files for each component. If you take the file manager (file-mgr ZNode), it has 2 child nodes as mentioned to store properties files and other configuration files.

What are the other child nodes (etc, policy) in the ZNode structure?

This is where the design comes into play. Suppose we have a mime-types.xml file for our file manager. If you have configured file manager manually, you may have seen that there are many directories within OODT distribution (file-mgr, res-mgr, workflow and etc.) Within these directories (say within file-mgr directory) there are another set of directories like bin, lib, policy and etc. As you assume, bin and lib include all the libraries and executables. etc includes the major configuration files (filemgr.properties) and policy also includes several other files required for configuration purposes (ex: cmd-line-actions.xml). Therefore, when we do distributed configuration management, we have to make sure that all the instances which are downloading configuration from zookeeper will get all of these properties files and configuration files and they will store them in the correct directories for corresponding components to pickup these files in run time.

In order to make sure that all the configuration files will be available within a predefined directory at runtime, we store each configuration file in a ZNode whose path is same as the path where that file should be at runtime. If we take the mime-types.xml file, it should be available (locally) within the ${FILEMGR_HOME}/etc/ directory. Therefore, to identify where the corresponding configuration file should be stored locally relative to ${FILEMGR_HOME}, we take the ZNode path from ZNode file-mgr/configuration/ as the storing location. Therefore, when being downloaded, content within the ZNode oodt/components/file-mgr/configuration/etc/mime-types.xml will be stored in ${FILEMGR_HOME}/etc/mim-types.xml file. That is the basic idea of how configuration will be published and how they will downloaded and stored.

DistributedConfigurationPublisher is responsible for publishing configuration to zookeeper initially. Once configuration has been published, any OODT component running in any cluster node can fetch them through DistributedConfigurationManager class. A CLI tool is available to publish/verify/clear configuration in zookeeper. To learn more on configuration publishing, please read the documentation on Distributed Configuration Management.