How to setup centralized logging for Airavata

If you want to enable elasticsearch based logging feature for Airavata

Centralized logging for Airavata

This is a new feature we are currently working on 0.17 relea

se so the feature is not yet released. We have added a custom log appender for Airavata and its not registered by default. If you enable it it will push all the Json formatted logs to a Kafka topic. So if you want to enable this

feature you first have to setup a Kafka cluster so that all the logs will be pushed to the Kafka topic created based on the configuration provided. Once you have kafka cluster setup then you can start airavata by simply changing two parameters in airavata-server.properties, but if you want to know

how to setup Kafka cluster please use below related articles for the instructions.

How to enable KafkaAppender

Overwrite the following property names with your configurations based on your Kafla setup.

# Kafka Logging related configuration
isRunningOnAws= false - Set to true if you are running Airavata on AWS
kafka.broker.list= localhost:9092 - One or more kafka broker node address with the port, Giving one is enough because KafkaProducer will find out the addresses of other nodes
kafka.topic.prefix= staging - Topic prefix you want to use because Airavata will create the topic names for you
enable.kafka.logging= true - Enable kafka Appender to register as a log Appender.

How is a sample Json log message look like ?

{
        "serverId" => {
        "serverId" => "192.168.59.3",
        "hostName" => "192.168.59.3",
         "version" => "airavata-0.16-135-gac0cae6",
           "roles" => [
            [0] "gfac"
        ]
    },
           "message" => "Skipping Zookeeper embedded startup ...",
         "timestamp" => "2016-09-09T20:57:08.329Z",
             "level" => "INFO",
        "loggerName" => "org.apache.airavata.common.utils.AiravataZKUtils",
              "mdc"  => {
      				"gateway_id": "21845d02-7d2c-11e6-ae22-562311499611",
      				"experiment_id": "21845d02-7d2c-11e6-ae22-34b6b6499611",
     				"process_id": "21845d02-7d2c-11e6-ae22-56b6b6499611",
      				"token_id": "21845d02-7d2c-11e6-ae22-56b6b6499611"
    			},
        "threadName" => "main",
          "@version" => "1",
        "@timestamp" => "2016-09-09T20:57:11.678Z",
              "type" => "gfac_logs",
              "tags" => [
        [0] "local",
        [1] "CoreOS-899.13.0"
    ],
    "timestamp_usec" => 0
}

How airavata create kafka topic names by given topic prefix

Airavata has few services and its completely flexible for you to deploy in the way you like, you can deploy all the services (Apache thrift services) in on JVM or you can create one JVM for each component or you can merge only few of them to one JVM. In the above log you can see the roles section contains only gfac which means above log was taken from a gfac server node and no other component was running in that JVM. So topic creation logic is based on the role of the JVM. To keep the deployment clean we recommend to deploy one component per JVM so that its easier to scale

and diagnose the system. So during topic creation we check the number of roles configured in the JVM

if the number of roles is greater than 4 => <kafka_topic_prefix>_all_logs ex: staging_all_logs (kafka.topic.prefix = staging)

Otherwise we pick the first role => <kafka_topic_prefix>_<first role>_logs ex: staging_gfac_logs (kafka.topic.prefix = staging)

Space shortcuts

Page tree