Page History

...

Logstash Configuration on the log shipping layer.

=====================================================================================================
input {
  file {    
type => "apache"    
path => [ "/var/log/cloudstack/management/management-server.log" ]
  }
}
output {
  amqp {
    stdout { codec => rubydebug }
redis { host => "myamqpserver"
    exchange192.168.56.100" data_type => "fanout"
    name list" key => "rawlogslogstash" }
  }
}}
=====================================================================================================

Logstash configuration on the index/search layer.

input {
  amqp redis {
    type host => "all"
    host <host>"
# these settings should match the output of the agent
data_type => "<host>list"
    exchange key => "rawlogs"
    name =>  "<name>"
  logstash"

# We use the 'json' codec here because we expect to read
# json events from redis.
codec => json
}
}
filter
{
grok
   grok {
     type match => [ "apache"
     pattern => message","%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY}[T ]%{HOUR}\:?%{MINUTE}\:?%{SECOND}[T ]INFO%{GREEDYDATA}job[\-]+%{INT\:jobid}\s*=\s*\[\s*%{UUID\:uuid}\s*
\]%{GREEDYDATA}"
  ]
named_captures_only => true
}
}
output
{
  stdout { debug => true debug_format => "json"}
elasticsearch {
    host => "<elasticsearch_master><host>"
  }
}

=====================================================================================================

API Command :

A new API command ExtractLogByJobIdCmd will be introduced. This will be implemented as a synchronous command.

...