...

• scope of this proposal is restricted to achieving distributed routing and network acl's with OpenVswitch in VPC
• scope of this proposal is restricted to OpenVswtich integration on XenServer/KVM
• enhancements called out in [8] for efficient handling of ARP/DHCP traffic and preventing unicast storms are not in scope of this proposal and functional specification
• scope of this proposal is restricted to achieving distributed routing/acl's with out-of-the box openvswitch in xenserver/kvm.  

Glossary & Conventions

OVS:/OpenvSwitch. Open vSwitch[5] is a production quality, multilayer virtual switch designed to enable massive network automation through programmatic extension

...

table=5, priority=1000,nw_src=10.1.2.0/24 actions=mod_dl_src=mac address for 10.1.2.1, modl_dl_dst=mac address for destination VM,output:1

Packet flows

...

Lets consider few packet flows to understand how logical router and flow rules achieve distributed routing.

...

• introduce new 'Connectivity' service capability 'distributedrouting'. This capability shall indicate 'Connectivity' service providers ability to perform distributed routing & ACL's.
• createVPCOffering API shall be enhanced to take 'distributedrouting' as capability for 'Connectivity' service.
• Provider specified for the 'Connectivity' service shall be validated with capabilities declared by the service provider, to ensure provider supports 'distributedrouting' capability.
• listVPCOfferings API shall return VpcOfferingResponse response that shall contain 'Connectivity' service's  'distributedrouting' capability details of the offering if it is configured
• createNetworkOffering API shall throw exception if distributedrouting' capability is specified for 'Connectivity' service. 

...

• hosts on which a network (for tier in VPC) spans
• hosts on which VPC spans (cumulative of host on which individual tier's in VPC spans)
• list of VPC's that span on the host

OVS tunnel manager enhancements

...

• keep track of state of the tunnels for a network between two hosts for a tier in VPC
• state of logical router on a host for a VPC
• function to tell if 'logical router' can be enabled on a host for a VPC
• enable logical router on a host for VPC
• disable logical router on a host for VPC
• back ground thread that periodically performs
  • get the list of VPC's that has distributed routing enabled
    • for each VPC in the list
      • get the list of hosts on which VPC scansspans
        • check the state of tunnels from the host and toward the host 
        • if the tunnel is not established attempt to establish tunnel
        • if all the tunnels established enable logical router

...

• list of hosts on which VPC spans
• state(enabled/disabled) of logical router on a host for the VPC
• state of tunnels between the hosts for a tier in the VPC

UI changes

• createVpcOffering APi shall have ability to create VPC offering with 'distributedrouting' as connectivity service capability. No UI is needed for this change (unless there is plan to add UI for createVpcOffering) since there is no UI for VPC offering creation itself. 
• In the VPC details view, there shall be action to view the current state of tunnel between the hosts on which VPC spans

Convergence time

Following are the events that require state update of configuration and corresponding latency: 

• on first VM launch/ last VM destroy on a host need setting up/destroy tunnel mesh for each tier's bridge with rest of the hosts. Latency to replicate configuration is proportional to  (n*(n-1))/2 * m where  n is the number of hosts on which VPC spans and m is the number of tiers in the VPC. n*(n-1)/2
• on VPC tier create/delete, needs setting up/destroy of bridge for the tier on all the hosts and setting up tunnel mesh. Latency is proportional to n * n-1/2 where n is number of hosts on which VPC spans + setting up the router entry on logical router
• on replace network ACL for a tier needs update of acl table on logical router on each host.

Performance and scaling issues

...