Problem

There are multiple parties like event-driven workflows systems, monitoring system, usage & billing system would be interested in getting notification of a events in the cloud in near instantaneous manner. In CloudStack event could be state change of virtual or psychical resources, an action performed by user (action events), policy based events (alerts). Currently CloudStack has no notification mechanism to let the interested parties know of the occurrence of an event. Polling by invoking listEvents and listAlerts is the only way to know occurrence of events. Also there are no externsion hooks one can implement to get the event notification. Though there is StateListener interface which has both pre and post state transition event call back methods, only user VM resource has state listener implemented. Most of the resorces volume, network, nic, etc has states but does not have state machine and StateListener. So for a component interested getting notification need to get tightly coupled into orchestration workflow. For e.g, code to generate usage date (usage events) is spread across the orchestration components.

Proposal

This proposal is to introduce an in-process event bus in the management server and by which achieve event notifications. Scope of the event bus is only with in the management server, so only CloudStack components and extension plug-in's can be subscribers to the events. Goal of this proposal is to have bare minimum mechanism in the CloudStack code, using which more powerful notification mechanism (like SMS, email etc) can be built as CloudStack extensions.

Ideal requirements for such event bus is to be

  • light weight (a library)
  • provide simple interface
  • decouple the publisher and subscriber so that they need not be aware of other
  • powerful enough to provide filters for events of a type to wildcard (all events)
  • deliver events asynchronously. Publisher publishes the event and forgets.

In order for CloudStack be able to generate rich set of events and have clean mechanism to generate events, its important to have state machine associated with every resource that is managed by CloudStack. So another goal of this proposal is to associate with every resource (User VM, Volume, Nic, Network, public IP, snapshot, template etc) with a state machine and generate events as part of the state change. So the publishers of events in CloudStack will become that StateListeners associated with the resource. 

For e.g. UserVMStateListener which is listener for the state changes of user VM's would create event object and post on to the event bus as below.

    public boolean postStateTransitionEvent(State oldState, Event event, State newState, VirtualMachine vo, boolean status, Object opaque) {

            _eventBus.post(new UserVmStateChangeEventVO(....));

     }

A subscriber interested in the user VM state change would register to the event and get the events as below

    _eventBus.register(this)

    @Subscribe

    public void userVMStateChangetHandler(UserVmStateChangeEventVO event)

Use cases

Following are some use cases demonstrated in the diagram below

  • Usage/billing engines: a 3rd party Cloud Usage solution can implement a plug-in to CloudStack that subscribes to  CloudStack events and generates usage data, that is consumed by their usage software
  • AMQP plugin can put all the events on the a message queue, a AMQP message broker then can provide topic based notification to subscribers
  • pub/sub notification service can be implemented as Pluggable service in to CloudStack that provides rich API for event notifications (topics based subscription and notification) and deal with multi-tenancy, authentication&authorization issues

Discussion topics

  • Google Guava Event bus library which is under Apache License 2.0 seems to be good fit for implementing event bus in CloudStack. Its light-wieght and has simple interface but very powerful. But any other alternatives could be evaluated
  • If there is clustered management server setup then subscribers need to be present on the both management servers. Using a messaging queue servers like RabbitMQ can be used for event bus, but i thinks will add complexity into CloudStack. Are there any scenarios where having in-process event bus proposed will fall short in clustered management server setups?
  • No labels