THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Bug Reference
https://issues.apache.org/jira/browse/CLOUDSTACK-9163
Branch
4.8
Introduction
Currently the inter VPC traffic has to go thru the public gateway. This means the traffic has to be nat-ed across public internet via core-routers, which is inefficient in itself. A more efficient approach will be to route the traffic within cloudstack and even better if no nating is involved.
OSPF provides a way to connect two VPCs using the optimal route between them without needing nat-ing. OSPF achieves this by maintaining and advertising the most efficient routes between various OSPF enabled routers. When a new VPC is added its OSPF enabled router advertises the routes to other routers, thereby each of them learn to route traffic properly between them.
This design document looks into how OSPF can be implemented in cloudstack to make inter VPC communication more manageable and efficient.
Implementation
In this implementation the focus will be on IPv4, though Quagga supports both IPv4 and IPv6 and will not be constrained when in future IPv6 support is added. The firewall, port forwarding, Network ACLs, DNS, DHCP and Password Reset services running on the router will continue to work as they do now on VPC routers.
When OSPF is selected for a zone, cloudstack will prompt for additional information in order to configure a inter VPC-Transit network on the existing public network for this zone. For simplicity we will use the existing zone wide public network as VPC-transit network. A VPC VR router sits at the boundary of the VPC tiered network in Area X. This router will advertise its routing table to the other Area X routers including the designated router. These routes are then picked up by other boundary routers. When one of the VPC's VM wants to communicate with another VPC's VM the router now knows how to direct the traffic via the VPC/Transit network(which is also the cloudstack public network).
All the VPC VR will be connected to transit network forming OSPF Area X. Cloudstack will auto generate the priorities of these routers that will help them elect a designated router.
OSPF will take care of routing the traffic originating from various tiers to another VPCs tiers. So that none of the existing functionality is affected all the other static routes will still be set on quagga based router. (An example of static routes is attached at the end of this doc)
Limitation: In this first implementation, 1. only one OSPF area per Zone will be supported, 2. the implementation will be for IPv4 only.
About Quagga
Quagga implements OSPF (v2, v3) and is found suitable to run on debian based VPC virtual router to provide dynamic routing. Quagga is an advanced software routing package that provides a suite of TCP/IP-based routing protocols and provides implementations of OSPFv2, OSPFv3, RIP v1 and v2, RIPng and BGP-4 for Unix-like platforms, particularly FreeBSD, Linux, Solaris and NetBSD.
OSPF Transit Network
The public network (VPC-Transit network) will be used as transit network for OSPF for all the dynamically routed VPCs in the zone. If dynamic routing is enabled for the the zone additional information will be collected. This information will be stored in network_details table for the given public network.
Following additional information is required to enable dynamic routing for a zone:
Field Name | Description |
|---|---|
| Protocol | Dropdown to select the protocol: OSPF or BGP |
| OSPF Area | Specify the OSPF Area ID |
| Hello Interval | Set number of seconds for HelloInterval timer value. Setting this value, Hello packet will be sent every timer value seconds on the specified interface. This value must be the same for all routers attached to a common network. The default value is 10 seconds. |
| Dead Interval | Set number of seconds for RouterDeadInterval timer value used for Wait Timer and Inactivity Timer. This value must be the same for all routers attached to a common network. The default value is 40 seconds. |
| Retransmit Interval | Set number of seconds for RxmtInterval timer value. This value is used when retransmitting Database Description and Link State Request packets. The default value is 5 seconds. |
| Transmit Delay | Set number of seconds for InfTransDelay value. LSAs’ age should be incremented by this value when transmitting. The default value is 1 seconds. |
| Authentication | Dropdown with 2 options: MD5 or Text Plain |
| Password | The password for OSPF Area |
| Super-CIDR** | For using dynamic routing a zone level super-cidr is required. This will be carved into network sub levels and into the created routed tiers. |
**Super-CIDRs
We need a super cidr that will be used to allocate cidrs to tiers that will be part of dynamic routing. Initial implementation will have this super-cidr available to all domains and accounts. The ip addresses will be distributed as per RFC 3531.
Lets say user enters 200.200.0.0/16, this can be split into 256 subnets using /24. Which are 200.200.0.0/24 to 200.200.255.0/24. This means a total of 256 VPCs can be created in one single zone.
Then each 200.200.100.0/24 will be further split into 8 network 200.200.100.0/27. This means that first /27 network is for VPC and the next 7 for the VPC tiers.
Initial implementation will split the Super IPv4 CIDR to /24 networks and each /24 network will be split into /27 networks. If the super IPv4 CIDR is not sufficiently big enough to split into network sub levels then appropriate errors will be given to the user.
The last ip address of the VPC level super-cidr will be used as the loopback id of the ospf enabled VPC-VR.
Some of the operators may want their users to have public IP assigned to their VPC-VMs. To facilitate that management server will allow both private and public class super-cidrs to be specified for the zone. There will be provision to add more super-CIDRs of private public kind at later stages.
Routed VPC Network Offering
There will be two default service offerings that will be created for dynamically routed VPCs one corresponding to public class ips and another corresponding to private class ips. These will be "DefaultPublicRoutedVPCNetworkOffering" and "DefaultPrivateRoutedVPCNetworkOffering" respectively. Depending on what offering a user chooses cloudstack will assign ips from the respective range to user's VPC or flag an error if no ips of that particular class exists.
These offerings will show up with other default network offerings. When a Routed VPC is created it will spin of the VPC-VR with pre-configured ranges and quagga service running on it. This will require that the system vm template have quagga installed on them.
VR-template changes
Quagga will be pre-installed on the VPC-VR template and will be activated and configured if Default<Private/Public>RoutedVPCNetworkOffering network offering is used. Quagga will advertise the VPC routing table across to other VPC routers.
[ APPENDIX - Quagga configuration]
Creating Routed VPC
In the network tab when VPC is selected then UI will show a additional button to create routed VPCs as
The user while creating a VPC can select the Default<Private/Public>RoutedVPCNetworkOffering that will enforce the Super IPv4 CIDR to be in the Super IPv4 CIDR of the zone. If the user does not provide any value then the cloudstack will automatically pick a /24 CIDR as the super CIDR for this VPC. An appropriate error will be given if the zone has exhausted all /24 CIDRs.
Creating routed VPC tier
When a VPC tier is created, the user shall configure an IPv4 CIDR and the IPv4 gateway. The tier IPv4 CIDR should be within the super IPv4 CIDR configured for its VPC. In this case it would be /27 prefix. If the user does not configure any value, Cloudstack automatically picks an unused /27 CIDR and assigns it to the VPC tier.
The workflow for VPC creation will look as below:
The Routed VPC creation and configuration will be similar to the creation of a regular VPC, only that the Super CIDRs of various tiers are carved from the zone's configured Super CIDR for dynamically routed VPCs.
DB Schema changes
Following schema objects will be added to the MS schema:
network_details: The zone level dynamic routing parameters for public network will be saved in network details table.
network_offering: the table will be modified and a new field dynamic_routing will be added to it.
user_vm_details: for capturing priority of the OSPF router in order to facilitate selection of designated router.
Quagga Configuration
Quagga Router VPC-VR1 (hostname: r-6-VM)
#
interface eth0
description link to link local network
ip address 169.254.3.148/24
link detect
interface eth1
description link to public network
ip address 192.168.100.68/24
link detect
interface eth2
description link to VPC tier1
ip address 200.100.100.0/27
interface eth3
description link to VPC tier 2
ip address 200.100.100.32/27
interface lo
ip address 200.100.100.255
link detect
hostname r-6-VM
router ospf
network 200.100.100.0/27 area 0
network 200.100.100.32/27 area 0
network 169.254.0.0/24 area 0
network 182.168.100.0/24 area 0
line vty
no login
enable password <password>
Static Routes:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.100.1 0.0.0.0 UG 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
## Not set 200.100.100.0 0.0.0.0 255.255.255.224 U 0 0 0 eth2
## Not set 200.100.100.32 0.0.0.0 255.255.255.224 U 0 0 0 eth3
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
Overview
Content Tools
Apps