Dynamically routed VPC

Bug Reference

https://issues.apache.org/jira/browse/CLOUDSTACK-9163

Branch

4.8

Introduction

  In cloudstack traffic between VPC’s is routed via there public interface by core-routers. OSPF protocol running on VPC routers in conjunction with the core-routers will take care of routing between different VPC’s as well as public network in most efficient manner.  VPC’s traffic will be directly routed via its router without going thru the core routers. 

Use Case

In large organisations where the end-to-end access is a requirement, the current VPC model brings more complexity to management and hands-on configuration beside the overhead involved in the NAT address translations processing. In this kind of enterprise environments, the VR needs to route networks using both, public and private IPs. Using this approach, the NATing is unnecessary because the networks will be routed and dynamically announced using the OSPF protocol. When a new network is added, OSPF protocol advertises it to the Core  Router. Neighbouring VPC routers will then get this advertisement from the Core Routers. This way dynamically routed VPC router will be aware of such routers in their neighbourhood and will be able to route the traffic directly to such routers.

The VPC's thus form an autonomous system with core routers sitting at the boundary of this autonomous system.

About Quagga

  Quagga implements OSPF (v2, v3) and is found suitable to run on debian based VPC virtual router to provide dynamic routing. Quagga is an advanced software routing package that provides a suite of TCP/IP-based routing protocols and provides implementations of OSPFv2, OSPFv3, RIP v1 and v2, RIPng and BGP-4 for Unix-like platforms, particularly FreeBSD, Linux, Solaris and NetBSD.

Implementation

In this implementation the focus will be on IPv4, though Quagga supports both IPv4 and IPv6 and will not be constrained when in future IPv6 support is added. The firewall, port forwarding, Network ACLs, DNS, DHCP and Password Reset services running on the router will continue to work as they do now on VPC routers.

When OSPF is selected for a zone, cloudstack will prompt for additional information in order to configure a inter VPC-Transit network on the existing public network for this zone.  For simplicity we will use the existing zone wide public network as VPC-transit network. A VPC VR router sits at the boundary of the VPC tiered network in Area X. This router will advertise its routing table to the other Area X routers including the designated router. These routes are then picked up by other boundary routers. When one of the VPC's VM wants to communicate with another VPC's VM the router now knows how to direct the traffic via the VPC/Transit network(which is also the cloudstack public network).

All the VPC VR will be connected to transit network forming OSPF Area X. Cloudstack will auto generate the priorities of these routers that will help them elect a designated router.

So that none of the existing functionality is affected all the existing static routes will also be set on quagga router.

Limitation: In this first implementation, 1.  only one OSPF area per Zone will be supported, 2. the implementation will be for IPv4 only.

OSPF Transit Network

The public network (VPC-Transit network) will be used as transit network for OSPF for all the dynamically routed VPCs in the zone. If dynamic routing is enabled for the the zone additional information will be collected. This information will be stored in network_details table for the given public network.

Following additional information is required to enable dynamic routing for a zone:

**Super-CIDRs 

We need a super cidr that will be used to allocate cidrs to tiers that will be part of dynamic routing. Initial implementation will have this super-cidr available to all domains and accounts. The ip addresses will be distributed as per RFC 3531.

Lets say user enters 200.200.0.0/16, this can be split into 256 subnets using /24. Which are 200.200.0.0/24 to 200.200.255.0/24. This means a total of 256 VPCs can be created in one single zone. 

Then each 200.200.100.0/24 will be further split into 8 network 200.200.100.0/27. This means that first /27 network is for VPC and the next 7 for the VPC tiers.

Initial implementation will split the Super IPv4 CIDR to /24 networks and each /24 network will be split into /27 networks. If the super IPv4 CIDR is not sufficiently big enough to split into network sub levels then appropriate errors will be given to the user.

Routed VPC Network Offering

A new default service offering "DefaultRoutedVPCNetworkOffering" for routed networks will be added in the management server. This will show up with other default network offerings. When a Routed VPC is created it will spin of the VPC-VR with pre-configured quagga service running on it. This will require that the system vm template have quagga installed on them.

VR-template changes

Quagga will be pre-installed on the VPC-VR template and will be activated and configured if DefaultRoutedVPCNetworkOffering network offering is used.  (Using cloud-early-config.sh).  Quagga will advertise the VPC routing table across to other VPC routers. 

[ APPENDIX - Quagga configuration]

Creating Routed VPC

In the network tab when VPC is selected then UI will show a additional button to create routed VPCs as

The user while creating a VPC can select the DefaultRoutedVPCNetworkOffering that will enforce the Super IPv4 CIDR to be in the Super IPv4 CIDR of the zone. If the user does not provide any value then the cloudstack will automatically pick a /24 CIDR as the super CIDR for this VPC. An appropriate error will be given if the zone has exhausted all /24 CIDRs.

Creating routed VPC tier

When a VPC tier is created, the user shall configure an IPv4 CIDR and the IPv4 gateway. The tier IPv4 CIDR should be within the super IPv4 CIDR configured for its VPC. In this case it would be /27 prefix. If the user does not configure any value, Cloudstack automatically picks an unused /27 CIDR and assigns it to the VPC tier. 

The workflow for VPC creation will look as below:

The Routed VPC creation and configuration will be similar to the creation of a regular VPC, only that the Super CIDRs of various tiers are carved from the zone's configured Super CIDR for dynamically routed VPCs.