h1. Introduction
h2. Purpose
This is functional specification for Syslog Enhacements feature of Cloudstack.
h2. References
[https://issues.apache.org/jira/browse/CLOUDSTACK-712]
h2. Useful Links
[Brief Introduction to Syslog|http://en.wikipedia.org/wiki/Syslog]
h2. Use cases
These Syslog alert messages can be used by remote Syslog Servers to find out the problems in Cloudstack and can act accordingly. This is similar to alerts that we see on dashboard of Cloudstack.
# Admin will set the Ip of remote Syslog hosts through Cloudstack configuration file, log4j-cloud.xml and those remote Syslog Hosts will start getting Syslog alert messages then.
# Admin can delete the Syslog Hosts by removing them from configuration file
h1. Feature Specifications
In this feature we are implementing the Syslog enhancements for alerts in CloudStack. In the current scenario we are writing alerts to database and are shown on Cloudstack dashboard. Now in addition to that we will also send the syslog messages in a format which can be easily analyzed by external Syslog messages analyzer.
We will send the Syslog messages for following alerts
# *availableMemory* : Available Memory below configured threshold
# *availableCpu* : Unallocated CPU below configured threshold
# *availableStorage* : Available Storage below configured threshold
# *remainingStorageAllocated* : Remaining unallocated Storage is below configured threshold
# *unallocatedVirtualNetworkpublicIp :* Number of unallocated virtual network public IPs is below configured threshold
# *unallocatedP{*}{*}rivateIp* : Number of unallocated private IPs is below configured threshold
# *availableSecondaryStorage* : Available Secondary Storage in availability zone is below configured threshold
# *host :* host related alerts like host disconnected etc
# *userVmState :* User VM stopped unexpectedly
# *domainRouterVmState :* Domain Router VM stopped unexpectedly
# *consoleProxyVmState :* Console Proxy VM stopped unexpectedly
# *routingConnection :* lost connection to default route (to the gateway)
# *storageIssueSystemVms :* storage issue in system VMs
# *usageServerStatus :* No usage server process running
# *managmentNode :* Management network CIDR is not configured originally
# *domainRouterMigrate :* Domain Router VM Migration was unsuccessful
# *consoleProxyMigrate :* Console Proxy VM Migration was unsuccessful
# *userVmMigrate :* User VM Migration was unsuccessful
# *unallocatedVlan :* Number of unallocated VLANs is below configured threshold in availability zone
# *ssvmStopped :* SSVM stopped unexpectedly
# *usageServerResult* : Usage job failed
# *storageDelete :* Failed to Delete storage pool
# *updateResourceCount :* Failed to update the resource count
# *usageSanityResult :* Usage Sanity Check failed
# *unallocatedDirectAttachedPublicIp* : Number of unallocated shared network IPs is low in availability zone
# *unallocatedLocalStorage :* Remaining unallocated Local Storage is below configured threshold
# *resourceLimitExceeded :* Resource limit exceeds the limit
Each message will contain the following keys and corresponding value
# alertType
# message
# podId
# dataCenterId
# clusterId
h4. Message structure for Syslog messages will be as follows
_Date severity_level Management_Server_IP_Address/Name alertType:_: value _dataCenterId_:: value _podId_:: value _clusterId_:: value _message_:: value
if some keys are not valid(like dataCenterId is 0) then those will not be set
For Example
Mar 4 10:13:47 WARN localhost alertType:: *managmentNode* message:: Management server node 127.0.0.1 is up
Mar 4 10:13:47 WARN 10.1.1.1 alertType:: *managmentNode* message:: Management network CIDR is not configured originally. Set it default to 10.144.6.0/23
currently sending all the alerts to administrator and he will filter alerts according to his needs.
For this feature, we will be using log4j library with Apache License
h2. Architecture and Design description
This feature will be implemented as plugin. This feature will use the log4j Appender to get the required alerts and will generate the Syslog messages from that. SyslogAppender of log4j will be used to send the Syslog messages
h3. Classes
We will add following classes
* *AlertsSyslogAppender* contains mainly two methods
** *append(LoggingEvent) :* sends the Syslog message to the configured Syslog Hosts using SyslogAppender class
** *setSyslogAppenders() :* sets the Syslog Appenders to whom for each different Syslog Host
h3. Changes in log4j-cloud.xml
To configure multiple Syslog Hosts one needs to add following appender in log4j-cloud.xml
<appender name="ALERTSYSLOG">
<param name="Threshold" value="WARN"/>
<param name="SyslogHosts" value=""/>
<param name="Facility" value="LOCAL6"/>
<layout>
<param name="ConversionPattern" value="%-5p \[%c{3}\] (%t:%x) %m%n"/>
</layout>
</appender>
To specify multiple Syslog Hosts one has to modify as follows with each Syslog Host separated by ,
<appender name="ALERTSYSLOG">
<param name="Threshold" value="WARN"/>
<param name="SyslogHosts" value="10.1.1.1,10.1.1.2"/>
<param name="Facility" value="LOCAL6"/>
<layout>
<param name="ConversionPattern" value="%-5p \[%c{3}\] (%t:%x) %m%n"/>
</layout>
</appender>
Following loggers will also be added
<logger name="com.cloud.alert.AlertManagerImpl" additivity="false">
<level value="WARN"/>
<appender-ref ref="SYSLOG"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
<appender-ref ref="ALERTSYSLOG"/>
</logger>
<logger name="com.cloud.usage.UsageAlertManagerImpl" additivity="false">
<level value="WARN"/>
<appender-ref ref="SYSLOG"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
<appender-ref ref="ALERTSYSLOG"/>
</logger> |