You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

High-level view of the steps required for implementing a basic connectivity (L2 networking) manager for CloudStack

The aim of this document is to provide an overview of the work required to extend CloudStack’s networking infrastructure in order to allow folks to plug new networking solutions into CloudStack. This document is only concerned with providing basic connectivity, i.e. layer-2 networking.

Overview

CloudStack has a modular set of interfaces and a dynamic configuration management system, which allows folks to replace the traditional way in which CloudStack manages guest networks. It is intended to further evolve these interfaces to make this integration easier.

Relevant Components

This section introduces the components of CloudStack’s virtual networking infrastructure, which might be implemented in order to allow folks to plug new solutions into CloudStack.

Network Guru

Network Gurus are responsible for:

  • Design and implementation of virtual networks
  • IP Address Management

From com.cloud.network.guru.NetworkGuru JavaDoc:A Network goes through the following life cycles through the NetworkGuru.

  • When a guest network is created, NetworkGuru is asked to "design" the network.
    This means the NetworkGuru checks the parameters such as cidr, gateway,
    vlan, etc and returns a network that can work with those paremeters.
    Note that at this point the network is only a virtual network. It has
    not been substantiated with resources, such as vlan, to make the network
    functional in the physical environment. At this stage, the network is in
    Allocated state.
  • When the first virtual machine is about to be started and requires network
    services, the guest network needs to have resources to make it usable
    within the physical environment. At this time, the NetworkGuru is
    called with the implement() method to acquire those resources.
  • For every virtual machine starting in the network, the NetworkGuru is
    asked via the reserve() method to make sure everything the virtual
    machine needs to be functional in the network is reserved.
  • For every virtual machine being stopped in the network, the NetworkGuru
    is informed via the release() method to make sure resources occupied
    by the virtual machine is released.
  • If all virtual machines within the network have been stopped, the guest
    network is garbage collected. When a guest network is garbage collected
    the NetworkGuru is informed via the shutdown() method to release any
    resources it allocated to that network.
  • When a guest network is being deleted, the NetworkGuru is informed via
    the trash() method.It is worth noting that CloudStack loads at start up every guru declared in components.xml; however some of them might be exclusive. For instance, the guru for managing VLAN based guest networks cannot work with the guru for managing L2-in-L3 networks. In this case it is up to the guru itself to ensure it does not clash with other gurus. Generally, it is expected that the cloud administrator will configure components.xml to avoid this clash. In the future, CloudStack plans to allow for selection of NetworkGuru based on the isolation method supported by the underlying physical network.  (i.e. If it is VLAN, CloudStack will pick GuestNetworkGuru but if it is OVS, it will pick OvsNetworkGuru, etc.).

Network Element

Network Elements represent components that are present in a CloudStack network. Such components can provide any kind of network service or support the virtual networking infrastructure and their interface is defined by com.cloud.network.element. NetworkElement.

Examples of network elements are: the virtual router, security groups, external load balancers (NetScaler, BigIP, SRX), as well as external DHCP servers, and also Open vSwitch (in some cases).

Among the methods exposed by this interface it is worth mentioning the following ones:

  • Implement
    Configures the specified network on a given network element. This method is invoked only if the network element is associated with a service provider in the network offering.
  • isReady

Verifies whether the current instance of the network element is ready to be used

  • Prepare

This method prepares the network element for adding a new NIC to the network. This method is quite important with network elements for basic connectivity (see com.cloud.network.element.OvsElement), whereas it is typically left unimplemented for network element providing higher-layer services, such as load balancing.

  • Release

Dual of the prepare method.

  • getProvider

Returns the service provider associated with the current instance of the Network Element. For instancecom.cloud.network.element.ExternalDhcpElement is associated with the ‘ExternalDHCPServer’ provider.

Network Managers

Not to be confused with CloudStack networking manager, which implements the interface com.cloud.network.NetworkManager, they are in charge of handling the resources managed by the network elements. They typically extend the genericcom.cloud.utils.component.Manager interface, and are therefore loaded at startup through CloudStack’s configuration manager.

For instance, the manager for setting up L2-in-L3 networks with Open vSwitch is com.cloud.network.ovs.OvsTunnelManagerImpl, whereas Virtual Router lifecycle is managed by com.cloud.network.router.VirtualApplianceManagerImpl.

Resource

The resource class abstract the physical (or virtual) resource being managed. CloudStack defines several resource classes, including hypervisors as well as Load Balancers and Storage devices. Resources are manager classes as well, as they implement thecom.cloud.resource. ServerResource interface which extends the Manager.

The most important method of a Resource class is probably executeRequest which is invoked by the agent manager1. This method is typically implemented with a cascaded if statement which invokes a different method according to the command dispatched by the agent manager.

CloudStack networking flows

This section discusses some operational flows regarding CloudStack networking. This is only as far as basic connectivity is concerned; details about virtual router, security groups, and external network services, such as NetScaler or SRX, are outside the scope of this document.

Network creation process

The following is, in brief, the flow within CloudStack when a new guest network is created. Relevant points for integration with additional network managers are highlighted in red.

  • The command in the API layer invokes CloudStack’s network service (com.cloud.network.NetworkService, implemented bycom.cloud.network.NetworkManagerImpl)
  • The Network Service validates Network Offering, and the physical network, if specified
  • Verify the account has administrative rights for shared networks
  • Validate IP information for the network (IP interval, netmask, and gateway)
  • Validate CIDR (User’s CIDR limit, overlapping CIDRs) and VLAN id (if specified)
  • Create a network data model object (com.cloud.network.NetworkVO) and fill it with domain, CIDR, gateway, broadcast domain type (e.g.: VLAN, vSwitch) and broadcast URI.
  • Iterate over network gurus and invoke the design method
    • Method accepts a Network Offering, a Deployment Plan, a Network instance and the owner Account

Network setup process upon VM startup

The following is, in brief, the sequence of operations CloudStack performs for setting up networking when a new instance is started. Relevant points for integration with additional network managers are highlighted in red.

  • During the VM start-up process, the prepare method is invoked on the Network Service instance
  • retrieve and sort NICs for VM, then for each NIC:
    • Retrieve data model object for NIC’s network
    • Find network guru and invoke implement method on it
      • Does whatever is required to configure the network on virtual/physical networking resouces
    • Configure network (CIDR, Gateway, etc.)
    • Retrieve list of providers for the Network Offering. For each Network Element in providers list, implement the element invoking the implement method on the com.cloud.network.element.NetworkElement instance
    • Note: The implementation process is skipped if the network is already implemented or currently being implemented in another thread
    • Fetch NIC’s broadcast URI (usually same as the network’s broadcast URI) and create NIC profile
    • Invoke the reserve method on the Network Guru.
      • Set IP addresses, MAC, broadcast, netmask, and gateway for NIC
    • Ask network elements to prepare for the new NIC, invoking the prepare method on each network element.
      • Does whatever is required to make sure the newly created NIC has connectivity across all the network elements in the current network offering
    • Add entry (entries) for NIC into the DHCP service provider and the UserData service providers; mark whether NIC supports security groups or not
    • Add NIC to VM profile

Implementing the Components

This section contains some notes concerning the implementation of the previously discussed interfaces in order to allow additional virtual networking solutions to plug into CloudStack.

Network Guru

It is worth starting from the class com.cloud.network.guru.GuestNetworkGuru, which should be regarded as the base class for all gurus dealing with guest networks.

Overriding the behaviour of the network guru becomes necessary if the default IP configuration behaviour must be overridden. Also the network guru may override:

  • vNet allocation, e.g.: for using GRE keys, or multicast groups identifiers in place of VLAN tags for identifying guest networks;
  • Network implementation: for redefining CIDR assignation or customize the strategy for setting the network’s broadcast URI

Network Element

Each appliance, physical or virtual, participating in some form in the network should get its own element. For instance, when OVS is used for building L2-in-L3 network, it is regarded as a network element. In a scenario where an external software component is used for building and managing virtual network, it makes sense for this component to be modelled as a network element in CloudStack.

In order to plug a new network manager into CloudStack, an the implementation of the NetworkElement interface might override:

  • The implement method for preparing the configuration for a new network;
  • The prepare method for setting up all the required configuration for a new NIC;
  • The release method for destroying configuration specific to a given NIC being removed.

Network Managers

Network managers can be implemented in order to provide an interface which reflects the peculiarities of the new solution being plugged into CloudStack, while the interface implemented by the network element reflects CloudStack’s process for setting up networks. For instance, the OVS network element implements the prepare method for setting up the network for a given NIC; this method invokes the CheckAndCreateTunnel method on the OVS Tunnel Manager, which in turns dispatches commands to the resource where OVS is running (XenServer or KVM hypervisor).

Network managers are not constrained to a particular interface, apart from the generic Manager interface. Several manager components in CloudStack networking subsystem define completely different interfaces. Please note that these components are not strictly necessary, as in theory the network element could communicate directly with the managed resource.

Resources

A new network manager could define as many resources as required. CloudStack already provides several resource components which can be used as guidance for implementing them.

As the final purpose of a resource is to execute a command dispatched by the agent manager, resource implementers should also define command classes, which are usually POJOs extending the abstract class com.cloud.agent.api.Command. For each command Class an answer class should also be defined. The answer class should extend com.cloud.agent.api.Answer (which also is a concrete realization of the previously introduced Command class)

Resources and Network Managers are a matched pair.  Network  Manager is responsible for OAM&P logic running inside the management server and has access to the database to help it maintain state.  Resource is a translation layer that might be running co-located with the physical network resource and may not have access to the database and should not maintain state other than what is on the physical network resource itself.

Just like managers, resources are not strictly necessary. In theory a Network Element could implement a client for the API of the new controller and therefore be completely self-contained. 2

Loading the components into CloudStack

When new components are added (managers, elements, resources, gurus) they must be loaded into CloudStack. The easiest way for doing so is to declare them in CloudStack component configuration files and let the ‘Component Locator’ instantiate them.

Components can be declared in the following way:

  1. Edit client/tomcatconf/components.xml.in
  2. Find the _adapters _key for Network Gurus:

<adapters key="com.cloud.network.guru.NetworkGuru">

  1. Add entries for the new network guru(s) like the following:

<adapter name="AnotherGuestNetworkGuru"/>

  1. Find the adapters key for Network Elements

<adapters key="com.cloud.network.element.NetworkElement">

  1. Add entries for the new network element(s) like the following:

<adapter name="Another"/>

  1. Repeat similar steps for network managers and resources, if any.

An alternative to editing the components.xml.in file is to extend a component library and specify the extended component library in components.xml.in file:

<management-server library="com.cloud.configuration.AnotherComponentLibrary">

Note: Most of the components loaded by CloudStack are defined in the components libraries, which can be found in the packagecom.cloud.configuration (server.jar)

Once declared either in components.xml.in or in the component library, components can be referenced using the @Inject annotation, which will load the instance for the component loaded from the component locator.

Moreover, it might be necessary to add global configuration parameters for enabling the new network manager. Similar parameters have already been defined for OVS network managers. It is important to note that the code must check the value of these parameters in order to ensure gurus/elements/managers are not triggered for the wrong kind of network.

Another possibility worth being considering is to:

  • Describing basic connectivity as a service provider (requires CloudStack code changes)
  • Define a network offering which maps the element for the new network manager to the basic connectivity provider
  • Choose the newly created network offering when creating guest networks.

1� The discussion of agent managers, resources, and command dispatching in CloudStack is unfortunately beyond the scope of this document.
2� Integrating with the agent-resource framework might provide anyway several benefits in terms of scalability and coverage for unit tests.

  • No labels