https://issues.apache.org/jira/browse/CLOUDSTACK-9163
In cloudstack traffic between VPC’s is routed via there public interface by core-routers. OSPF protocol running on VPC routers in conjunction with the core-routers will take care of routing between different VPC’s as well as public network in most efficient manner. VPC’s traffic will be directly routed via its router without going thru the core routers.
In large organisations where the end-to-end access is a requirement, the current VPC model brings more complexity to management and hands-on configuration beside the overhead involved in the NAT address translations processing. In this kind of enterprise environments, the VR needs to route networks using both, public and private IPs. Using this approach, the NATing is unnecessary because the networks will be routed and dynamically announced using the OSPF protocol. When a new network is added, OSPF protocol advertises it to the Core Router. Neighbouring VPC routers will then get this advertisement from the Core Routers. This way dynamically routed VPC router will be aware of such routers in their neighbourhood and will be able to route the traffic directly to such routers.
The VPC's thus form an autonomous system with core routers sitting at the boundary of this autonomous system.
Quagga implements OSPF (v2, v3) and is found suitable to run on debian based VPC virtual router to provide dynamic routing. Quagga is an advanced software routing package that provides a suite of TCP/IP-based routing protocols and provides implementations of OSPFv2, OSPFv3, RIP v1 and v2, RIPng and BGP-4 for Unix-like platforms, particularly FreeBSD, Linux, Solaris and NetBSD.
In this implementation the focus will be on IPv4, though Quagga supports both IPv4 and IPv6 and will not be constrained when in future IPv6 support is added. The firewall, port forwarding, Network ACLs, DNS, DHCP and Password Reset services running on the router will continue to work as they do now on VPC routers.
When OSPF is selected for a zone, cloudstack will prompt for additional information in order to configure a inter VPC-Transit network on the existing(?) public network for this zone. A VPC VR router sits at the boundary of the VPC identified by a unique area. This router will advertise its routing table to the Area 0 router which is a core router and also a area boundary router. These routes are then picked up by other boundary routers. When one of the VPC's VM wants to communicate with another VPC's VM the router now knows how to direct the traffic via the VPC/Transit network(which is also the cloudstack public network).
The public network (VPC-Transit network) will be used as transit network for OSPF for all the dynamically routed VPCs in the zone. If dynamic routing is enabled for the the zone additional information will be collected. This information will be stored in network_details table for the given public network.
Following additional information is required to enable dynamic routing for a zone:
Field Name | Description |
---|---|
Protocol | Dropdown to select the protocol: OSPF or BGP |
OSPF Area | Specify the OSPF Area ID |
Hello Interval | Set number of seconds for HelloInterval timer value. Setting this value, Hello packet will be sent every timer value seconds on the specified interface. This value must be the same for all routers attached to a common network. The default value is 10 seconds. |
Dead Interval | Set number of seconds for RouterDeadInterval timer value used for Wait Timer and Inactivity Timer. This value must be the same for all routers attached to a common network. The default value is 40 seconds. |
Retransmit Interval | Set number of seconds for RxmtInterval timer value. This value is used when retransmitting Database Description and Link State Request packets. The default value is 5 seconds. |
Transmit Delay | Set number of seconds for InfTransDelay value. LSAs’ age should be incremented by this value when transmitting. The default value is 1 seconds. |
Authentication | Dropdown with 2 options: MD5 or Text Plain |
Password | The password for OSPF Area |
Super-CIDR** | For using dynamic routing a zone level super-cidr is required. This will be carved into network sub levels and into the created routed tiers. |
We need a super cidr that will be used to allocate cidrs to tiers that will be part of dynamic routing. Initial implementation will have this super-cidr available to all domains and accounts. The ip addresses will be distributed as per RFC 3531.
Lets say user enters 200.200.0.0/16, this can be split into 256 subnets using /24. Which are 200.200.0.0/24 to 200.200.255.0/24. This means a total of 256 VPCs can be created in one single zone.
Then each 200.200.100.0/24 will be further split into 8 network 200.200.100.0/27. This means that first /27 network is for VPC and the next 7 for the VPC tiers.
Initial implementation will split the Super IPv4 CIDR to /24 networks and each /24 network will be split into /27 networks. If the super IPv4 CIDR is not sufficiently big enough to split into network sub levels then appropriate errors will be given to the user.
A new default service offering "DefaultRoutedVPCNetworkOffering" for routed networks will be added in the management server. This will show up with other default network offerings. When a Routed VPC is created it will spin of the VPC-VR with pre-configured quagga service running on it. This will require that the system vm template have quagga installed on them.
Quagga will be pre-installed on the VPC-VR template and will be activated and configured if DefaultRoutedVPCNetworkOffering network offering is used. (Using cloud-early-config.sh). Quagga will advertise the VPC routing table across to other VPC routers.
[ APPENDIX - Quagga configuration]
Creating Routed VPC
In the network tab when VPC is selected then UI will show a additional button to create routed VPCs as
The user while creating a VPC can select the DefaultRoutedVPCNetworkOffering that will enforce the Super IPv4 CIDR to be in the Super IPv4 CIDR of the zone. If the user does not provide any value then the cloudstack will automatically pick a /24 CIDR as the super CIDR for this VPC. An appropriate error will be given if the zone has exhausted all /24 CIDRs.
When a VPC tier is created, the user shall configure an IPv4 CIDR and the IPv4 gateway. The tier IPv4 CIDR should be within the super IPv4 CIDR configured for its VPC. In this case it would be /27 prefix. If the user does not configure any value, Cloudstack automatically picks an unused /27 CIDR and assigns it to the VPC tier.
The workflow for VPC creation will look as below:
The Routed VPC creation and configuration will be similar to the creation of a regular VPC, only that the Super CIDRs of various tiers are carved from the zone's configured Super CIDR for dynamically routed VPCs.
Following schema objects will be added to the MS schema:
network_details: The zone level dynamic routing parameters for public network will be saved in network details table.
network_offering: the table will be modified and a new field dynamic_routing will be added to it.
interface eth0
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 ospf
router ospf
ospf router-id 200.0.0.11
log-adjacency-changes
redistribute connected
passive-interface default
no passive-interface eth0
network 200.0.0.0/24 area 0.0.0.0
area 0.0.0.0 authentication message-digest
interface eth0
ipv6 ospf6 instance-id 0
!
router ospf6
router-id 200.0.0.11
redistribute connected
interface eth0 area 0.0.0.0
interface eth0
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 ospf
router ospf
ospf router-id 200.0.0.12
log-adjacency-changes
redistribute connected
passive-interface default
no passive-interface eth0
network 200.0.0.0/24 area 0.0.0.0
area 0.0.0.0 authentication message-digest
interface eth0
ipv6 ospf6 instance-id 0
!
router ospf6
router-id 200.0.0.12
redistribute connected
interface eth0 area 0.0.0.0