Purpose

This is the functional specification for the supporting multiple physical networks for guest traffic over VMware deployments which has Jira Reference CLOUDSTACK-7151. This will also address the cluster level traffic override functionality breakage (due to fix for CLOUDSTACK-5884) of vmware traffic labels defined at the cluster.

Bug Reference

CLOUDSTACK-7151 - vmware: Type of vSwitch was not detected correctly while preparing public/guest virtual networks.

Branch

master

Introduction

The support for cluster level override settings (Introduced in 4.2.0 with VMware DVS Integration) for guest and public traffic types blocks support for multiple physical networks defined in the zone for these traffic types. Particularly guest traffic, which needs higher scale compared to other traffic types, may need multiple physical networks. Hence it is required to support multiple physical networks defined in the zone for these traffic types at zone and cluster level without impacting existing cluster level override settings. Also, renaming of vSwitch name (SVS/DVS) for guest and public traffic type overridden at cluster in Cloudstack deployment has become troublesome, requiring manual update to database. This need to be addressed such that admin would be able to seamlessly re-name vSwitch name (SVS/DVS) at cluster in Cloudstack whenever required even in production cloud.

Document History

Glossary

• DVS - VMware vNetwork Distributed Virtual Switch
• SVS - VMware vNetwork Standard Virtual Switch
• Nexus 1000v - Cisco Nexus 1000v Distributed Virtual Switch

Usecases

1. Admin wants to use multiple physical networks for a traffic type at zone or cluster level. This enables the service providers to create different guest networks with different network parameters (Ex: 1G vs 10G, highly redundant network fabric, etc.) and tag this to appropriate network offerings.
2. Admin wants to rename DVS/SVS overridden at the cluster, being used by specific traffic type after deploying zone.

Feature specification

1. Support multiple physical networks for a traffic type at zone and cluster level.
2. Support renaming a virtual switch (Standard vSwitch OR DVS OR Nexus 1000v ethernet port profile) overridden at cluster using traffic label.

Test Guidelines

1. Create a zone where guest traffic span across more than 1 physical network.
2. Create a zone with SVS/DVS/Nexus 1000v instance as backend and try re-naming the switch name after making sure that there exists some instance, which is actively under use by 1 or more instances and system vms.

Error Handling

1. If admin specifies invalid (non-existent) name for SVS/DVS/Nexus 1000v instance, then report the incident, warn the user and continue to use existing instance.

Audit Events

1. Events will be generated in the management server logs for any resource (for e.g. VM, NIC) being created during the course of the deployment.
2. vCenter tasks involved during a vSwitch and dvPortGroup create/update operations while creating CloudStack virtual networks.

Target Users

1. CloudStack Admins.

Current behavior

1. Cluster level traffic override settings (Introduced in 4.2.0 with VMware DVS Integration) for guest and public traffic types has blocked the support for multiple physical networks in the zone for these traffic types. These settings are required for customers looking to span the physical networks in zone across different vSwitch (any of the 3 types) instances or a combination of vSwitch types. For example, a zone has been using only SVS and admin want to add new cluster which is to use only DVS or vice versa.
2. The hierarchy of the traffic level settings applied for vSwitch name and type:
   1. Cluster level traffic override settings would take higher precedence when defined. The admin should define the actual vSwitch type and name for overriding the traffic at cluster level. If not, traffic settings would fallback to Zone level settings.
   2. Zone level traffic settings would be the next precedence after cluster level. The admin should clearly define the vSwitch type, name and vlan-id(if required). If not, traffic settings to fallback to Cloud level settings.

   3. Cloud/Global level settings would be the default if zone level settings are not defined. These settings would be based on the global configuration parameters "vmware.use.dvswitch" and "vmware.use.nexus.vswitch". The vSwitch type and name based on these parameters configuration is listed in the below table. All these default vSwitch names are hardcoded. 

      vmware.use.dvswitch	vmware.use.nexus.vswitch	Valid Configuration?	vSwitch Type	vSwitch Name
      true	true	Yes	Cisco Nexus Distributed vSwitch	epp0
      true	false	Yes	VMware Distributed vSwitch	dvSwitch0
      false	true	No	VMware Standard vSwitch	vSwitch0
      false	false	Yes	VMware Standard vSwitch	vSwitch0

      
      

3. The cluster level traffic override settings introduced in 4.2.0 were broken in 4.3 due to fix for CLOUDSTACK-5884. In 4.3, the vSwitch type and name were picked from the zone level traffic settings ignoring the traffic settings overridden at the cluster. This issue is tracked under CLOUDSTACK-7151.
   

Design description

1. Support multiple physical networks per traffic at zone and cluster level. Existing cluster level override settings does support only single physical network. This needs to be improved to support N physical networks for a traffic type.
   1. Possible alternative solutions:
   1. Remove cluster level traffic override settings (supported for guest and public only) from this version onwards.
   1. This ensures for all newly created zones to support multiple physical networks.
   2. For existing deployments based on older versions, where at least one cluster using cluster level traffic override settings, Cloudstack does not support multiple physical networks per traffic.
   3. For existing deployments based on older versions, where no cluster exists that uses cluster level traffic override settings, perform data migration over table cluster_details removing the overridden settings such that Cloudstack support multiple physical networks per traffic
   4. This also implicitly solves traffic label/vswitch re-name issue. No further changes required to support seamless re-name.
   2. Add support for multiple physical networks in cluster level traffic override setttings.
   1. While adding cluster, provide means to override (guest/public) traffic and specify vswitch type for that traffic and vswitch label per each of the physical network discovered in zone for that traffic type.
   2. Persist vswitch label for each physical network per each traffic (guest/public) for each cluster in database, i.e. cloud.cluster_physical_network_traffic_info.
   3. Persist vswitch type of guest and public traffic in cloud.cluster_details when admin chooses to override traffic while adding cluster.
   4. While implementing virtual networks over vSwitch, retrieve the vswitch type from table cloud.cluster_details and vswitch label from tablecloud.cluster_physical_network_traffic_info based on the physical network to be used for virtual network orchestration.
   2.  The solution (ii) above would be considered for implementation so that the existing customers with cluster level override settings were not impacted and the support for multiple physical networks at both zone and cluster level.
   3. Cloudstack never supported cluster level traffic override for management and storage traffic and this continues as is. No change would be done for management and storage traffic as part of this feature.
      
      
2. Support renaming a virtual switch (Standard vSwitch OR DVS OR Nexus 1000v ethernet port profile) overridden at cluster with running user instances (production cloud) et. al.
   1. For existing clusters with override settings in place - Check if old label (zone level) is matches with vswitch settings in cloud.cluster_details table. If settings are matching, then update the new label, i.e. vswitch type and/or name as applicable per new label specified by admin, in cloud.cluster_details table along with cloud.physical_network_traffic_types table.
   2. For new clusters or existing cluster without override settings - Update table cloud.cluster_details and cloud.physical_network_traffic_types reflecting new switch name and type values at cluster for specific physical network when defined.

Limitations

1. Cloudstack does not support more than 1 DVS per physical network.

API changes

1. addCluster API call needs additional parameters for overridden vswitch name in VMware environment.
   1. New parameter physicalnetworktrafficlabels is added. This is a map of physical network traffic id (uuid for each traffic type in physical_network_traffic_types table) and network label (vswitch name) for specifying overridden vswitch name for each of public and guest traffic type per each physical network in zone.
   2. Existing parameters guestvswitchname and publicvswitchname are still supported for backward compatibility. These parameters would override the first guest and public traffic vswitch name in the physical networks of the zone and the vswitch names of these traffic types in other physical networks (if any) are not addressed. The paramaters guestvswitchname and publicvswitchname would be deprecated in the future releases.
   3. physicalnetworktrafficlabels would take precedence over guestvswitchname and publicvswitchname for first guest and public traffic when that traffic id is specified in the physicalnetworktrafficlabels map also.
   4. Example: clustername=testcluster&guestvswitchtype=vmwaredvs&publicvswitchtype=vmwaredvs&physicalnetworktrafficlabels[0].physicalnetworktrafficid=30bb36bd-fe11-47ee-8101-4a3090fb3aba&physicalnetworktrafficlabels[0].networklabel=dvSwitchTest0&physicalnetworktrafficlabels[1].physicalnetworktrafficid=3ba3eba1-0f1a-4611-b58e-8bf90b1d206f&physicalnetworktrafficlabels[1].networklabel=dvSwitchTest0 
   5. All these parameters are optional.
2. updateCluster API call needs additional parameters for renaming the vswitch type.
   1. New parameter physicalnetworktrafficlabels is added. This is a map of physical network traffic id (uuid for each traffic type in physical_network_traffic_types table) and network label (vswitch name) for renaming the overridden vswitch name for each of public and guest traffic type per each physical network in zone.
   2. Example: id=14470ed5-9df0-4acb-a3b1-4144def4ddc9&physicalnetworktrafficlabels[0].physicalnetworktrafficid=30bb36bd-fe11-47ee-8101-4a3090fb3aba&physicalnetworktrafficlabels[0].networklabel=dvSwitchNew1&physicalnetworktrafficlabels[1].physicalnetworktrafficid=3ba3eba1-0f1a-4611-b58e-8bf90b1d206f&physicalnetworktrafficlabels[1].networklabel=dvSwitchNew1
   3. All these parameters are optional.

DB changes

cluster_physical_network_traffic_info (id, uuid, cluster_id, physical_network_traffic_id, vmware_network_label) -  This table will contain the mapping between a physical network and switch it is associated with.

Hypervisors supported

VMware ESXi 5.0 and above.

UI Flow

Add cluster and update cluster forms should allow selection of vswitch type and display as many entries as the number of physical networks in the zone for updating guest/public vswitch name.

1. Add Cluster UI (in Add Cluster From and Add Zone Wizard) - when override traffic (guest/public) enabled, vSwitch Name text box for each physical network of guest and public in zone would be displayed for updating them.
2. Edit Cluster  (Cluster Details -> Edit Cluster)  - No edit functionality in cluster details now. Add edit functionality when guest/public traffic overridden and editing provision for vSwitch names for each physical network of guest/public overridden)

Upgrade

1. Data migration is required for all existing clusters, such that the vswitch name information per each traffic type (public and guest) is moved from existing table cluster_details to new table cluster_physical_network_traffic_info.

Open Items/Questions

N/A