You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

With the new distributed configuration management feature of OODT, users now get the chance deploy large number of instances of OODT components without having to worry about manually configuring all those instances. This new feature allows any OODT component to retrieve configuration required from zookeeper on the fly.

Example Scenario

For example, lets consider the file manager component. Suppose that we want to deploy 20 file manager instances in a cluster. Configuring them one by one would be a tiresome task with the user repeating the same set of tasks 20 times, copying properties files, XML files and such other files related to configuration of that file manager instance. If we had to configure a file manager instance manually, 

  • First, we need to get the OODT distribution zip file and unzip it. Then we need to modify the content of the configuration files (ex: etc/filemgr.properties) to match our requirement.
  • Then upload filemgr directory to the corresponding remote server.
  • Then we need to start the file manager process in the remote server.

To configure all 20 nodes, this process needs to be repeated 20 times. This becomes even harder if we had to customize configuration files specifically to each node in the cluster. Distributed Configuration Management feature is the solution for this. All the required configuration files first need to be stored in zookeeper prior to the deployment. Once the files are stored properly, users can copy the executables to the remote servers and run that component. When being initialized, this component will retrieve and store all the configuration files and properties files from zookeeper making all the required configuration files available locally at runtime. This becomes even more useful when users need to change some configuration at runtime and wants them to be reflected at each node in the cluster without manually uploading them again. Therefore, the new distributed configuration management feature is capable of listening to configuration changes in runtime and retrieve the corresponding changes without the users having to upload those files to each server manually.

For all these to work, first we need to store the corresponding configuration files in zookeeper. Distributed Configuration Publisher is the component responsible for storing and managing configuration information in zookeeper. Let's see how should we store configuration in zookeeper.

Distributed Configuration Publishing

As mentioned in the example above, we need to store all the configuration files in zookeeper prior to deploying our OODT components in servers. Following steps need to be taken to store configuration in zookeeper.

Setting up Zookeeper

First, we need to setup a zookeeper cluster as described in zookeeper documentation. For testing purposes, we can use the standalone zookeeper server. But for production deployments we need to setup a zookeeper cluster as described in that documentation. Please make sure to use zookeeper version higher than 3.5.

Zookeeper Connect String

Once you setup zookeeper, please note down the zookeeper connect string which will be used by our Distributed Configuration Publisher to connect to zookeeper. The connect string looks like IP1:PORT1,IP2:PORT2,...,IPn:PORTn where IP is the ip address of each zookeeper node and port is the client port on which each zookeeper server is listening on.

Set the environment variable ZK_CONNECT_STRING.

ex: export ZK_CONNECT_STRING=localhost:2181

Configuring configuration publisher

Now, download the latest distribution of apache OODT and extract it. There is a directory named config which contains the scripts required to publish configuration to zookeeper. Inside that directory will be etc, examples, lib directories and config-publisher executable. Out of those, etc contains log4j.xml for logging configuration and config-publisher.xml for configuring the configuration to be stored in zookeeper. examples contains some example configuration files for resource manager and file manager. For us to publish/store configuration in zookeeper, we need to modify the config-publisher.xml in the etc directory. You will find the following default content in that file out of the box.

 

config-publisher.xml
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
 
    <!-- File Manager example configuration publisher -->
    <bean id="filemgr-config-publisher" class="org.apache.oodt.config.distributed.DistributedConfigurationPublisher">

        <constructor-arg value="FILE_MANAGER"/>

        <!-- An optional project name, if you want to run several instances of the same component with different configuration -->
        <!--<constructor-arg value="project-x"/>-->

        <property name="propertiesFiles">
            <map key-type="java.lang.String" value-type="java.lang.String">
                <entry key="examples/filemgr/filemgr.properties" value="/etc/filemgr.properties"/>
                <entry key="examples/filemgr/logging.properties" value="/etc/logging.properties"/>
                <entry key="examples/filemgr/indexer.properties" value="/etc/indexer.properties"/>
                <entry key="examples/filemgr/mime-types.xml" value="/etc/mime-types.xml"/>
            </map>
        </property>

        <property name="configFiles">
            <map key-type="java.lang.String" value-type="java.lang.String">
                <!-- Examples - Core-->
                <entry key="examples/filemgr/core/elements.xml" value="/policy/core/elements.xml"/>
                <entry key="examples/filemgr/core/product-types.xml" value="/policy/core/product-types.xml"/>
                <entry key="examples/filemgr/core/product-type-element-map.xml" value="/policy/core/product-type-element-map.xml"/>
 
                <!-- Examples - Geo-->
                <entry key="examples/filemgr/geo/elements.xml" value="/policy/geo/elements.xml"/>
                <entry key="examples/filemgr/geo/product-types.xml" value="/policy/geo/product-types.xml"/>
                <entry key="examples/filemgr/geo/product-type-element-map.xml" value="/policy/geo/product-type-element-map.xml"/>
 
                <!-- Examples - Trace-->
                <entry key="examples/filemgr/trace/elements.xml" value="/policy/trace/elements.xml"/>
                <entry key="examples/filemgr/trace/product-types.xml" value="/policy/trace/product-types.xml"/>
                <entry key="examples/filemgr/trace/product-type-element-map.xml" value="/policy/trace/product-type-element-map.xml"/>
 
                <!-- Examples - DRAT-->
                <entry key="examples/filemgr/drat/elements.xml" value="/policy/drat/elements.xml"/>
                <entry key="examples/filemgr/drat/product-types.xml" value="/policy/drat/product-types.xml"/>
                <entry key="examples/filemgr/drat/product-type-element-map.xml" value="/policy/drat/product-type-element-map.xml"/>
            </map>
        </property>
    </bean>
</beans>

 

This is a spring configuration file where beans of type DistributedConfigurationPublisher needs to be defined for each OODT component you wish to deploy with the help of distributed configuration management feature. There is only one bean defined in the above example, which is responsible fore storing configuration related to file manager OODT component. Let's look at how this should be configured. 

  1. First define a bean of type org.apache.oodt.config.distributed.cli.DistributedConfigurationPublisher and give it a meaningful id. In the above example, we have given filemgr-config-publisher for the bean id. 

  2. Then set the constructor argument of that bean to the enum type of the OODT component you wish to publish configuration for. This value should be of enum type org.apache.oodt.config.Component. In the above example, it is org.apache.oodt.config.Component.FILE_MANAGER which stands for OODT File Manager. List of enum values that should be used per component is listed below. 

    Shorthand names for OODT components

    Following enum types need to be used (as constructor argument) when configuring the config-publisher.xml to publish configuration for any OODT component.

    OODT Component NameEnum types used when configuring config-publisher.xml
    File Managerorg.apache.oodt.config.Component.FILE_MANAGER
    Resource Managerorg.apache.oodt.config.Component.RESOURCE_MANAGER

  3. Now, we have to set the properties files to be stored in zookeeper for our selected OODT component. Since properties files and other configuration files need to be treated in a different manner at run time, properties files are considered separately. Following fragment from our example config-publisher.xml shows how we should define the configuration files to be stored in zookeeper.

    <property name="propertiesFiles">
	<map key-type="java.lang.String" value-type="java.lang.String">
		<entry key="examples/filemgr/filemgr.properties" value="/etc/filemgr.properties"/>
		<entry key="examples/filemgr/logging.properties" value="/etc/logging.properties"/>
		<entry key="examples/filemgr/indexer.properties" value="/etc/indexer.properties"/>
		<entry key="examples/filemgr/mime-types.xml" value="/etc/mime-types.xml"/>
	</map>
</property>

    Configuration takes a map of key value pairs where the key of each entry stands for the location (in your local machine) of the properties file to be stored in zookeeper. Value of each entry stands for the location where the corresponding properties file should be available once downloaded from zookeeper. For example,  

    <entry key="examples/filemgr/filemgr.properties" value="/etc/filemgr.properties"/>

      says that the file located at examples/filemgr/filemgr.properties (relative to oodt-distribution/config directory) needs to be available at ${OODT_COMPONENT_HOME}/etc/filemgr.properties location when being downloaded by the deployed instances.

  4. Similarly we have to setup the configuration files other than properties files to be stored in zookeeper. Example configuration fragment is shown below. 

    <property name="configFiles">
	<map key-type="java.lang.String" value-type="java.lang.String">
	    <!-- Examples - Core-->
	    <entry key="examples/filemgr/core/elements.xml" value="/policy/core/elements.xml"/>
	    <entry key="examples/filemgr/core/product-types.xml" value="/policy/core/product-types.xml"/>
	    <entry key="examples/filemgr/core/product-type-element-map.xml" value="/policy/core/product-type-element-map.xml"/>
	    <!-- Examples - Geo-->
	    <entry key="examples/filemgr/geo/elements.xml" value="/policy/geo/elements.xml"/>
	    <entry key="examples/filemgr/geo/product-types.xml" value="/policy/geo/product-types.xml"/>
	    <entry key="examples/filemgr/geo/product-type-element-map.xml" value="/policy/geo/product-type-element-map.xml"/>

	    <!-- Examples - Trace-->
	    <entry key="examples/filemgr/trace/elements.xml" value="/policy/trace/elements.xml"/>
	    <entry key="examples/filemgr/trace/product-types.xml" value="/policy/trace/product-types.xml"/>
	    <entry key="examples/filemgr/trace/product-type-element-map.xml"
		   value="/policy/trace/product-type-element-map.xml"/>

	    <!-- Examples - DRAT-->
	    <entry key="examples/filemgr/drat/elements.xml" value="/policy/drat/elements.xml"/>
	    <entry key="examples/filemgr/drat/product-types.xml" value="/policy/drat/product-types.xml"/>
	    <entry key="examples/filemgr/drat/product-type-element-map.xml" value="/policy/drat/product-type-element-map.xml"/>
	</map>
</property>

    As explained in the previous point, we have to give a mapping of local files to be stored in zookeeper along with the desired location of those files to be available relatively to ${OODT_COMPONENT_HOME} during run time. For example, 

    <entry key="examples/filemgr/core/product-types.xml" value="/policy/core/product-types.xml"/>

    says that the file located at examples/filemgr/oodt/product-types.xml (relative to bin directory) needs to be available at ${OODT_COMPONENT_HOME}//policy/oodt/product-types.xml location when being downloaded by the deployed instances. 

  5. We have now finished configuring the configuration publisher for file manager OODT component in the above steps. Similarly we can configure configuration publisher beans for other OODT components as per our requirement.

Publishing Configuration - Configuration Publisher CLI

We have now configured our config-publisher.xml with the required corresponding spring beans to match the OODT components we need to deploy with the help of distributed configuration management. Now we can use the Configuration publisher CLI  to actually publish/store the contents in the configuration files in zookeeper for deployed OODT components to download. The configuration publisher executable is located in the main conf directory and it is named conf-publisher. If you just run the conf-publisher as below without any arguments, it will print the usage as follows.

./conf-publisher
Starting Configuration Publisher
-----------------------------------------------------------------------------------------------------------------
| Short | Long                                             | Description
-----------------------------------------------------------------------------------------------------------------

 -a,     --action <action-name>                             This is the name of the action to trigger
 -c,     --config <arg>                                     Configuration publisher spring configuration XML
                                                              Requirement Rules: 
                                                               [publish : OPTIONAL, clear : OPTIONAL, verify : 
                                                               OPTIONAL] 

                                                              Handler: 
                                                               Will invoke 'setConfig' on action selected, except 
                                                               for the following actions: [publish : 
                                                               setConfigFile, verify : setConfigFile, clear : 
                                                               setConfigFile] 

 -cs,    --connectString <arg>                              Connect String to Zookeeper (ip1:port1,ip2:port2,..)
                                                              Requirement Rules: 
                                                               [publish : REQUIRED, clear : REQUIRED, verify : 
                                                               REQUIRED] 

                                                              Handler: 
                                                               Will invoke 'setConnectString' on action selected, 
                                                               except for the following actions: [publish : 
                                                               setConnectString, verify : setConnectString, clear 
                                                               : setConnectString] 

 -h,     --help                                             Prints help menu
 -psa,   --printSupportedActions                            Print Supported Actions
-----------------------------------------------------------------------------------------------------------------

As shown above, the CLI provides 3 major functions,

  • publish - Stores configuration specified in the config-publisher.xml in zookeeper
  • verify - Verifies whether the content stored in zookeeper are identical to the content in the configuration files (original files which we just published to zookeeper) stored locally.
  • clear - Removes any configuration data stored in zookeeper. This will delete only the configuration stored in zookeeper using the same config-publisher.xml. Therefore, users do not have to worry about all configuration being removed from zookeeper.

Also note that we need to give the zookeeper connect string with the -connectString option to the CLI so that the CLI will know to which zookeeper ensemble to connect to. Our next step is running the conf-publisher. Executing 

conf-publisher -connectString <zookeeper-connect-string> -a publish

in the bin directory will execute the configuration publisher and store all the configuration specified in zookeeper. Following is the possible output you will get.

ubuntu@ubuntu: bin$ ./conf-publisher -connectString 127.0.0.1:2181 -publish

 
Starting Configuration Publisher
 
log4j:WARN Continuable parsing error 53 and column 23
log4j:WARN The content of element type "log4j:configuration" must match "(renderer*,appender*,(category|logger)*,root?,categoryFactory?)".

INFO  DistributedConfigurationPublisher - Using zookeeper connect string : localhost:2181
INFO  DistributedConfigurationPublisher - Curator framework start operation invoked
INFO  DistributedConfigurationPublisher - Waiting to connect to zookeeper, startupTimeout : 30000
INFO  DistributedConfigurationPublisher - CuratorFramework client started successfully

Publishing configuration for : FILE_MANAGER
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/filemgr.properties to /projects/default/components/filemgr/properties/etc/filemgr.properties
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/logging.properties to /projects/default/components/filemgr/properties/etc/logging.properties
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/indexer.properties to /projects/default/components/filemgr/properties/etc/indexer.properties
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/mime-types.xml to /projects/default/components/filemgr/properties/etc/mime-types.xml
INFO  DistributedConfigurationPublisher - Properties files published successfully
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/core/elements.xml to /projects/default/components/filemgr/configuration/policy/core/elements.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/core/product-types.xml to /projects/default/components/filemgr/configuration/policy/core/product-types.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/core/product-type-element-map.xml to /projects/default/components/filemgr/configuration/policy/core/product-type-element-map.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/geo/elements.xml to /projects/default/components/filemgr/configuration/policy/geo/elements.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/geo/product-types.xml to /projects/default/components/filemgr/configuration/policy/geo/product-types.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/geo/product-type-element-map.xml to /projects/default/components/filemgr/configuration/policy/geo/product-type-element-map.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/trace/elements.xml to /projects/default/components/filemgr/configuration/policy/trace/elements.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/trace/product-types.xml to /projects/default/components/filemgr/configuration/policy/trace/product-types.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/trace/product-type-element-map.xml to /projects/default/components/filemgr/configuration/policy/trace/product-type-element-map.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/drat/elements.xml to /projects/default/components/filemgr/configuration/policy/drat/elements.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/drat/product-types.xml to /projects/default/components/filemgr/configuration/policy/drat/product-types.xml
INFO  DistributedConfigurationPublisher - Published configuration file examples/filemgr/drat/product-type-element-map.xml to /projects/default/components/filemgr/configuration/policy/drat/product-type-element-map.xml
INFO  DistributedConfigurationPublisher - Config files published successfully
Published configuration for : FILE_MANAGER

INFO  DistributedConfigurationPublisher - Configuration publisher destroyed
Exiting CLI ...

OK


Now we have published our configuration to zookeeper successfully.

Using Distributed Configuration Management

Once the configuration is published to zookeeper, we can deploy our OODT components to corresponding servers in the cluster. At this point, if we want to use the distributed configuration management in those instances, we have to follow the following steps.

  1. First, we should enable distributed configuration management by setting the OODT_DISTRIBUTED_CONF environment variable to true.

  2. Then,  we must either specify the zookeeper connect string as the ZK_CONNECT_STRING environment variable.

  3. When the above two steps are completed, we can run any OODT component with the support of distributed configuration management.

Example with file manager. We need to set the following two environment variables.

export OODT_DISTRIBUTED_CONF=true
export ZK_CONNECT_STRING=<zookeeper-connect-string>

When you run the file manager (complete oodt project may be) now, you will see a result similar to the following.

Using CATALINA_BASE:   /home/imesha/drat/deploy/tomcat
Using CATALINA_HOME:   /home/imesha/drat/deploy/tomcat
Using CATALINA_TMPDIR: /home/imesha/drat/deploy/tomcat/temp
Using JRE_HOME:        /opt/jdk
Using CLASSPATH:       /home/imesha/drat/deploy/tomcat/bin/bootstrap.jar:/home/imesha/drat/deploy/tomcat/bin/tomcat-juli.jar
Resource Manager started PID file (/home/imesha/drat/deploy/resmgr/run/cas.resmgr.pid).
Workflow Manager started PID file (/home/imesha/drat/deploy/workflow/run/cas.workflow.pid).
Tomcat started.
Aug 15, 2017 10:40:48 AM org.apache.oodt.cas.resource.system.XmlRpcResourceManager <init>
INFO: Loading Resource Manager Configuration Properties from: [/home/imesha/drat/deploy/resmgr/etc/resource.properties]
Aug 15, 2017 10:40:48 AM org.apache.oodt.cas.resource.system.XmlRpcResourceManager <init>
INFO: Resource Manager started by imesha
Aug 15, 2017 10:40:48 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [/home/imesha/drat/deploy/workflow/etc/workflow.properties]
log4j:WARN Continuable parsing error 53 and column 23
log4j:WARN The content of element type "log4j:configuration" must match "(renderer*,throwableRenderer?,appender*,plugin*,(category|logger)*,root?,(categoryFactory|loggerFactory)?)".
INFO  ConfigurationManagerFactory - Using distributed configuration management for FILE_MANAGER
INFO  DistributedConfigurationManager - Using zookeeper connect string : 127.0.0.1:2181
INFO  DistributedConfigurationManager - Curator framework start operation invoked
INFO  DistributedConfigurationManager - Waiting to connect to zookeeper, startupTimeout : 30000
INFO  DistributedConfigurationManager - CuratorFramework client started successfully
INFO  DistributedConfigurationManager - Properties loaded from ZNode at : /projects/default/components/filemgr/properties/etc/indexer.properties
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/etc/indexer.properties before writing new content
INFO  DistributedConfigurationManager - File from ZNode at etc/indexer.properties saved to /home/imesha/drat/deploy/filemgr/etc/indexer.properties
INFO  DistributedConfigurationManager - Properties loaded from ZNode at : /projects/default/components/filemgr/properties/etc/filemgr.properties
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/etc/filemgr.properties before writing new content
INFO  DistributedConfigurationManager - File from ZNode at etc/filemgr.properties saved to /home/imesha/drat/deploy/filemgr/etc/filemgr.properties
Aug 15, 2017 10:40:49 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: [null]
INFO  DistributedConfigurationManager - Properties loaded from ZNode at : /projects/default/components/filemgr/properties/etc/mime-types.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/etc/mime-types.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at etc/mime-types.xml saved to /home/imesha/drat/deploy/filemgr/etc/mime-types.xml
INFO  DistributedConfigurationManager - Properties loaded from ZNode at : /projects/default/components/filemgr/properties/etc/logging.properties
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/etc/logging.properties before writing new content
INFO  DistributedConfigurationManager - File from ZNode at etc/logging.properties saved to /home/imesha/drat/deploy/filemgr/etc/logging.properties
INFO  DistributedConfigurationManager - Properties loaded for : FILE_MANAGER
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/geo/elements.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/geo/elements.xml saved to /home/imesha/drat/deploy/filemgr/policy/geo/elements.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/geo/product-type-element-map.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/geo/product-type-element-map.xml saved to /home/imesha/drat/deploy/filemgr/policy/geo/product-type-element-map.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/geo/product-types.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/geo/product-types.xml saved to /home/imesha/drat/deploy/filemgr/policy/geo/product-types.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/core/elements.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/core/elements.xml saved to /home/imesha/drat/deploy/filemgr/policy/core/elements.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/core/product-type-element-map.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/core/product-type-element-map.xml saved to /home/imesha/drat/deploy/filemgr/policy/core/product-type-element-map.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/core/product-types.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/core/product-types.xml saved to /home/imesha/drat/deploy/filemgr/policy/core/product-types.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/trace/elements.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/trace/elements.xml saved to /home/imesha/drat/deploy/filemgr/policy/trace/elements.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/trace/product-type-element-map.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/trace/product-type-element-map.xml saved to /home/imesha/drat/deploy/filemgr/policy/trace/product-type-element-map.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/trace/product-types.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/trace/product-types.xml saved to /home/imesha/drat/deploy/filemgr/policy/trace/product-types.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/drat/elements.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/drat/elements.xml saved to /home/imesha/drat/deploy/filemgr/policy/drat/elements.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/drat/product-type-element-map.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/drat/product-type-element-map.xml saved to /home/imesha/drat/deploy/filemgr/policy/drat/product-type-element-map.xml
WARN  DistributedConfigurationManager - Deleting already existing file at /home/imesha/drat/deploy/filemgr/policy/drat/product-types.xml before writing new content
INFO  DistributedConfigurationManager - File from ZNode at policy/drat/product-types.xml saved to /home/imesha/drat/deploy/filemgr/policy/drat/product-types.xml
INFO  DistributedConfigurationManager - Configuration files saved for : FILE_MANAGER
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by imesha
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManagerFactory <init>
FINE: Collecting XML policies from the following directories:
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManagerFactory <init>
FINE: file:///home/imesha/drat/deploy/filemgr/policy/core
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManagerFactory <init>
FINE: file:///home/imesha/drat/deploy/filemgr/policy/geo
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManagerFactory <init>
FINE: file:///home/imesha/drat/deploy/filemgr/policy/trace
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManagerFactory <init>
FINE: file:///home/imesha/drat/deploy/filemgr/policy/drat
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.util.XmlStructFactory getProductType
WARNING: metadata node missing for product type : urn:oodt:GenericFile
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManager loadProductTypes
FINE: XMLRepositoryManager: found product type: [GenericFile]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManager loadProductTypes
FINE: XMLRepositoryManager: found product type: [LocationAwareProduct]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManager loadProductTypes
FINE: XMLRepositoryManager: found product type: [TraceableFile]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManager loadProductTypes
FINE: XMLRepositoryManager: found product type: [RatLog]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.repository.XMLRepositoryManager loadProductTypes
FINE: XMLRepositoryManager: found product type: [RatAggregateLog]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
INFO: Loading File Manager Configuration Properties from: [/home/imesha/drat/deploy/filemgr/etc/filemgr.properties]
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer setFileManagerUrl
INFO: Local Data Transfer to: [http://localhost:9000] enabled
Aug 15, 2017 10:40:50 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager <init>
INFO: File Manager started by imesha

You may have noted that the logs say "replacing existing file ...." several times. That is because, if the distributed configuration manager find a file with the same name at a location where a downloaded file to be stored, it will delete the existing file and create a new file with the downloaded content.

org.apache.oodt.config
  • No labels