Page History

High-level view of the steps required for implementing a basic connectivity (L2 networking) manager for CloudStack

The aim of this document is to provide an overview of the work required to extend CloudStack’s networking infrastructure in order to allow folks to plug new networking solutions into CloudStack. This document is only concerned with providing basic connectivity, i.e. layer-2 networking.

Overview

CloudStack has a modular set of interfaces and a dynamic configuration management system, which allows folks to replace the traditional way in which CloudStack manages guest networks. It is intended to further evolve these interfaces to make this integration easier.

Relevant Components

This section introduces the components of CloudStack’s virtual networking infrastructure, which might be implemented in order to allow folks to plug new solutions into CloudStack.

Network Guru

Network Gurus are responsible for:

...

From com.cloud.network.guru.NetworkGuru JavaDoc:A Network goes through the following life cycles through the NetworkGuru.

• When a guest network is created, NetworkGuru is asked to "design" the network.
  This means the NetworkGuru checks the parameters such as cidr, gateway,
  vlan, etc and returns a network that can work with those paremeters.
  Note that at this point the network is only a virtual network. It has
  not been substantiated with resources, such as vlan, to make the network
  functional in the physical environment. At this stage, the network is in
  Allocated state.

• When the first virtual machine is about to be started and requires network
  services, the guest network needs to have resources to make it usable
  within the physical environment. At this time, the NetworkGuru is
  called with the implement() method to acquire those resources.

• For every virtual machine starting in the network, the NetworkGuru is
  asked via the reserve() method to make sure everything the virtual
  machine needs to be functional in the network is reserved.

• For every virtual machine being stopped in the network, the NetworkGuru
  is informed via the release() method to make sure resources occupied
  by the virtual machine is released.

• If all virtual machines within the network have been stopped, the guest
  network is garbage collected. When a guest network is garbage collected
  the NetworkGuru is informed via the shutdown() method to release any
  resources it allocated to that network.

• When a guest network is being deleted, the NetworkGuru is informed via
  the trash() method.It is worth noting that CloudStack loads at start up every guru declared in components.xml; however some of them might be exclusive. For instance, the guru for managing VLAN based guest networks cannot work with the guru for managing L2-in-L3 networks. In this case it is up to the guru itself to ensure it does not clash with other gurus. Generally, it is expected that the cloud administrator will configure components.xml to avoid this clash. In the future, CloudStack plans to allow for selection of NetworkGuru based on the isolation method supported by the underlying physical network.  (i.e. If it is VLAN, CloudStack will pick GuestNetworkGuru but if it is OVS, it will pick OvsNetworkGuru, etc.).

Network Element

Network Elements represent components that are present in a CloudStack network. Such components can provide any kind of network service or support the virtual networking infrastructure and their interface is defined by com.cloud.network.element. NetworkElement.

...

Returns the service provider associated with the current instance of the Network Element. For instancecom.cloud.network.element.ExternalDhcpElement is associated with the ‘ExternalDHCPServer’ provider.

Network Managers

Not to be confused with CloudStack networking manager, which implements the interface com.cloud.network.NetworkManager, they are in charge of handling the resources managed by the network elements. They typically extend the genericcom.cloud.utils.component.Manager interface, and are therefore loaded at startup through CloudStack’s configuration manager.

For instance, the manager for setting up L2-in-L3 networks with Open vSwitch is com.cloud.network.ovs.OvsTunnelManagerImpl, whereas Virtual Router lifecycle is managed by com.cloud.network.router.VirtualApplianceManagerImpl.

Resource

The resource class abstract the physical (or virtual) resource being managed. CloudStack defines several resource classes, including hypervisors as well as Load Balancers and Storage devices. Resources are manager classes as well, as they implement thecom.cloud.resource. ServerResource interface which extends the Manager.

The most important method of a Resource class is probably executeRequest which is invoked by the agent manager1. This method is typically implemented with a cascaded if statement which invokes a different method according to the command dispatched by the agent manager.

CloudStack networking flows

This section discusses some operational flows regarding CloudStack networking. This is only as far as basic connectivity is concerned; details about virtual router, security groups, and external network services, such as NetScaler or SRX, are outside the scope of this document.

Network creation process

The following is, in brief, the flow within CloudStack when a new guest network is created. Relevant points for integration with additional network managers are highlighted in red.

• The command in the API layer invokes CloudStack’s network service (com.cloud.network.NetworkService, implemented bycom.cloud.network.NetworkManagerImpl)
• The Network Service validates Network Offering, and the physical network, if specified
• Verify the account has administrative rights for shared networks
• Validate IP information for the network (IP interval, netmask, and gateway)
• Validate CIDR (User’s CIDR limit, overlapping CIDRs) and VLAN id (if specified)
• Create a network data model object (com.cloud.network.NetworkVO) and fill it with domain, CIDR, gateway, broadcast domain type (e.g.: VLAN, vSwitch) and broadcast URI.
• Iterate over network gurus and invoke the design method
  • Method accepts a Network Offering, a Deployment Plan, a Network instance and the owner Account

Network setup process upon VM startup

The following is, in brief, the sequence of operations CloudStack performs for setting up networking when a new instance is started. Relevant points for integration with additional network managers are highlighted in red.

• During the VM start-up process, the prepare method is invoked on the Network Service instance
• retrieve and sort NICs for VM, then for each NIC:
  • Retrieve data model object for NIC’s network
  • Find network guru and invoke implement method on it
    • Does whatever is required to configure the network on virtual/physical networking resouces
  • Configure network (CIDR, Gateway, etc.)
  • Retrieve list of providers for the Network Offering. For each Network Element in providers list, implement the element invoking the implement method on the com.cloud.network.element.NetworkElement instance
  • Note: The implementation process is skipped if the network is already implemented or currently being implemented in another thread
  • Fetch NIC’s broadcast URI (usually same as the network’s broadcast URI) and create NIC profile
  • Invoke the reserve method on the Network Guru.
    • Set IP addresses, MAC, broadcast, netmask, and gateway for NIC
  • Ask network elements to prepare for the new NIC, invoking the prepare method on each network element.
    • Does whatever is required to make sure the newly created NIC has connectivity across all the network elements in the current network offering
  • Add entry (entries) for NIC into the DHCP service provider and the UserData service providers; mark whether NIC supports security groups or not
  • Add NIC to VM profile

Implementing the Components

This section contains some notes concerning the implementation of the previously discussed interfaces in order to allow additional virtual networking solutions to plug into CloudStack.

Network Guru

It is worth starting from the class com.cloud.network.guru.GuestNetworkGuru, which should be regarded as the base class for all gurus dealing with guest networks.

...

• vNet allocation, e.g.: for using GRE keys, or multicast groups identifiers in place of VLAN tags for identifying guest networks;
• Network implementation: for redefining CIDR assignation or customize the strategy for setting the network’s broadcast URI

Network Element

Each appliance, physical or virtual, participating in some form in the network should get its own element. For instance, when OVS is used for building L2-in-L3 network, it is regarded as a network element. In a scenario where an external software component is used for building and managing virtual network, it makes sense for this component to be modelled as a network element in CloudStack.

...

• The implement method for preparing the configuration for a new network;
• The prepare method for setting up all the required configuration for a new NIC;
• The release method for destroying configuration specific to a given NIC being removed.

Network Managers

Network managers can be implemented in order to provide an interface which reflects the peculiarities of the new solution being plugged into CloudStack, while the interface implemented by the network element reflects CloudStack’s process for setting up networks. For instance, the OVS network element implements the prepare method for setting up the network for a given NIC; this method invokes the CheckAndCreateTunnel method on the OVS Tunnel Manager, which in turns dispatches commands to the resource where OVS is running (XenServer or KVM hypervisor).

Network managers are not constrained to a particular interface, apart from the generic Manager interface. Several manager components in CloudStack networking subsystem define completely different interfaces. Please note that these components are not strictly necessary, as in theory the network element could communicate directly with the managed resource.

Resources

A new network manager could define as many resources as required. CloudStack already provides several resource components which can be used as guidance for implementing them.

...

Just like managers, resources are not strictly necessary. In theory a Network Element could implement a client for the API of the new controller and therefore be completely self-contained. 2

Loading the components into CloudStack

When new components are added (managers, elements, resources, gurus) they must be loaded into CloudStack. The easiest way for doing so is to declare them in CloudStack component configuration files and let the ‘Component Locator’ instantiate them.

...