Page History

...

Glossary

Feature Specifications

Use cases

Root Admin users can use this API to quickly identify the sequence of steps related to a particular Job.

QA can use this api to link the log messages related to automated test failures. put the relevant use case/stories to explain how the feature is going to be used/work

Architecture and Design description

The system will comprise of a log shipping layer. This layer will be responsible for collecting logs from each of the management server shipping them to a centralized place. In the current design we are proposing logstash as the shipping layer. It  It will be configured to use rabbitmq to ship individual log files to a centralized location.  Fluentd could be another option.

The shipping phase will interact with another layer called the indexer/search layer. This layer will also enable storing the logs in a format that will help in writing search queries. In the current implementation we are proposing use of logstash to recieve the individual log files and elasticsearch to search through these. Before logstash outputs the recieved messages to elasticsearch, it will apply a specific grok filter that will split the input messages into key value pairs. The key value pair will allow creation of search queries by (key,value). Via the elasticsearch REST api , search queries can be constructed for required jobid.

...

• Same as Management Server: Each management server will be configured to run as a elasticsearch node. This particular configuration though simple can impact the management server as the elasticsearch service can be memory and cpu intensive. One of the management servers will be configured as the master node.
• Seperate Elasticsearch nodes: In this configuration the systemvm template can be used to spawn elasticsearch nodes. The number of such nodes should be configurable via a global parameter. One of the  node will be designated as the master node.

TODO: Details on using systemvm for elasticsearch nodes.

Logstash Configuration on the log shipping layer.

...

A new API command ExtractLogByJobIdCmd will be introduced. This will be implemented as a synchronous command.

Manager:

The manager class will implement the actual functionality of querying elastic search for log messages that match the specified filters. For doing this the Elasticsearch REST api queries will be used. Post method will be used with elasticsearch DSL to specify the required query. DSL is quite flexible and in future if support is required to filter by time stamp and other values DSL would help achieve that with ease.
DSL query for searching logs by jobid

...

Web Services APIs

A new API has been will introduced which can be accessed as
http://<host>:8080/client/api?command=extractLogByJobId&jobid=<jobid>

UI flow

None

IP Clearance

...

• Logstash which is an opensource log management tool and is covered under Apache 2.0 license
• Elasticsearch which is a search and analytics tool and is again covered under Apache 2.0 license. 

Appendix

Appendix A:Appendix B: 

Labels: