Internal Load Balancing between VPC tiers

Introduction

This documents gives an overview to the design and functional implementation for Internal Load Balancing on VPC tiers.

Feature developers:

• APIs and Business logic - Alena Prokharchyk
• Backend - TBA
• UI - TBA

Glossary

• Internal LB - the load balancing between internal (guest) cloudStack networks
• Internal LB Rule- Load balancer rule with the front end IP from Internal (Guest) cloudStack network.
• Internal LB VM - cloudStack system vm managing internal LB Rules

Use case

There are 2 tiers in the VPC - Web tier and Application tier. Traffic to Web tier is balanced on the VPC VR on the public side. User wants traffic coming from Web to the App tier to be balanced as well. Load balancing on the App tier will be covered by the Internal LB feature.

Internal LB can be handled by 2 network providers:

• InternalLBVm - have to add support for this provider first.
• Netscaler VPX

The pic below is for the case when InternalLBVm is used as a provider for the LB service on internal tier:

1. Public LB rule for 72.52.125.10 Public IP, public port 80 and private port 81. It enables LB for traffic coming from the internet to the vms on the Web tier. The LB rule is configured on the VPC VR.
2. Internal LB rule #1 for 10.10.10.4 guest IP, loadBalancerPort 23 and instancePort 25. The LB rule is configured on InternalLBVM1.
3. Internal LB rule #2 for 10.10.10.4 guest IP, loadBalancerPort 45 and instancePort 46. The LB rule is configured on InternalLBVM1.
4. Internal LB rule #3 for 10.10.10.6 guest IP, loadBalancerPort 23 and instancePort 25. The LB rule is configured on InternalLBVM2.

General flow

1) Enable Internal LB on VPC tier

In order to have Internal Load Balancing support on VPC tier, the tier has to be created from the network offering with:

• Service=LB, Provider=InternalLBVm.

or

• Service=LB, Provider=Netscaler, LB service capability "scheme=Internal".

Java code changes

• Network.java interface - add new capability for the LB service - supportedLbSchemes. It will have 2 values - Internal and Public.
• Existing Netscaler Service Provider will support both values - Internal and Public - for the supportedLbSchemes capability as it can act as a provider for Public and Internal Load Balancing.

• Introduce new Network Element - InternalLBVm. This provider supports only 1 service - LB, and its supportedLbMode capability can be Internal only. We need a new network element because Internal LB Vm will have different number of nics from the regular CS system vm (Guest and Control only); and its lifecycle will be different as well. The Vm will be started not when network gets implemented, but when the new IP gets acquired from the Guest Network for the Load Balancer rule.
• Introduce InternalLBNetworkApplianceManager - for managing Internal Load Balancer vms. 

Backend changes

TBD. We might need a separate template/set of scripts for Internal LB VM, or we can re-use the existing VR template.

Web Services API

list/createNetworkOffering APIs will return new parameter in the response - "internal_lb". The parameter type is Boolean

DB changes

No changes

2) Create Load Balancer Rule 

Problems to address:
* LB can be created for public ip address only, and it always references the ip by its db id (in WebServices API/DB/Everywhere in the code). As we maintain diff types of network ip addresses in separate datastructures and planning to support LB for different types of cloudStack ip addresses, we can no longer use this model.
* Web services LB API use "publicIpId"/"publicIp" namings in Request/Response parameters to address LB sourceIpAddress. As we are adding up new LB models - Internal in this release, perhaps some other mode in the future - we can't overload existing APIs with more parameters and split its logic based on the LB type as it will be too confusing for the API user.
* Currently LoadBalancer inherits from FirewallRule interface along with PF/StaticNat/Firewall/NetworkACL; all the rules are kept in the same datastructure. The rules are tight to the publicIpAddress. Changing LB to use ip/ipNetworkId, can affect other cloudStack rule types.
Refactoring:
* Separate LoadBalancer from the FirewallRule. The LB will no longer inherit from FirewallRule, will be maintained in the separate datastructure, and will be managed by the LoadBalancingRulesManagerImpl only. FirewallManager is not responsible for LB rules anymore.
Advantages - LB code changes will not affect other Network Rules and vice versa.
* Replace publicIpAddressId reference in the LB with sourceIp/sourceIpNetworkId pair. 
Advantages - opens the possibility to create LB rule for IP address of any cloudStack network.
* Introduce new set of Web Services APIs for the Load Balancing. The APIs can be used for both Public/Internal lb creation. "Scheme" - Internal/Public - parameter will reflect the type of the LB,all the rest of the parameters will be the same for all types of the LB (sourceIp, sourcePort, instancePort, etc). The new APIs are listed in the Internal lb FS:https://cwiki.apache.org/confluence/display/CLOUDSTACK/Internal+Load+Balancing+between+VPC+tiers
Advantages - if in the future we decide to support new type of LB - Elastic for example - all you have to change in the API is to add new supported value to "Scheme" parameter. On the backend, the scheme choice will determine the LB rule lifecycle and management.
Old LB APIs accepting publicIpId, will still be maintained by the cloudStack.

Problems to address

• LB can be created for public ip address only, and it always references the ip by its db id (in WebServices API/DB/Everywhere in the code). As we maintain diff types of network ip addresses in separate datastructures and planning to support LB for different types of cloudStack ip addresses, we can no longer use this model.
• Web services LB API use "publicIpId"/"publicIp" namings in Request/Response parameters to address LB sourceIpAddress. As we are adding up new LB models - Internal in this release, perhaps some other mode in the future - we can't overload existing APIs with more parameters and split its logic based on the LB type as it will be too confusing for the API user.
• Currently LoadBalancer inherits from FirewallRule interface along with PF/StaticNat/Firewall/NetworkACL; all the rules are kept in the same datastructure. The rules are tight to the publicIpAddress. Changing LB to use ip/ipNetworkId, can affect other cloudStack rule types.

Refactoring to be done

• Separate LoadBalancer from the FirewallRule. The LB will no longer inherit from FirewallRule, will be maintained in the separate datastructure, and will be managed by the LoadBalancingRulesManagerImpl only. FirewallManager is not responsible for LB rules anymore.

Advantages - LB code changes will not affect other Network Rules and vice versa.

• Replace publicIpAddressId reference in the LB with sourceIp/sourceIpNetworkId pair. 

Advantages - opens the possibility to create LB rule for IP address of any cloudStack network.

• Introduce new set of Web Services APIs for the Load Balancing. The APIs can be used for both Public/Internal lb creation. "Scheme" - Internal/Public - parameter will reflect the type of the LB,all the rest of the parameters will be the same for all types of the LB (sourceIp, sourcePort, instancePort, etc). 

Advantages - if in the future we decide to support new type of LB - Elastic for example - all you have to change in the API is to add new supported value to "Scheme" parameter. On the backend, the scheme choice will determine the LB rule lifecycle and management.

Old LB APIs accepting publicIpId, will still be maintained by the cloudStack.

Java code changes

• Add new manager - InternalLoadBalancerManagerImpl (name is TBD). This class will be responsible for managing Internal LB rules.
• The LBRule can be created with or without specifying the source IP address. If no source IP address is specified, the new one will get acquired automatically from the sourceIpNetworkId specified in createLoadBalancer command

If provider is InternalLBVM:

• A new InternalLBVM should be spanned as soon as new source IP address is acquired by the LB rule. This code should be handled by InternalLBNetworkApplianceManager. See more details on the VM in "InternalLBVM management and life cycle" section.
• There is going to be 1 Internal LB vm spanned per guest IP address participating in the Internal Load Balancing. If more rules are added for the same IP, they will be managed on the same InternalLBVM.

If LB provider is Netscaler VPX:

• TBD

Backend changes

TBD. For InternalLBVM, have to put HA proxy management/configuration details here

Web Services API

In the current code Load Balancer and Load Balancing Rule concepts are mixed up. We always represent it as a LoadBalancingRule to the end user, and Load Balancing Rule contains information about the load balancer itself - source LB Ip, name; and the actual LB rule (sourcePort/instancePort/protocol combination). With the new changes we are going to change the way we represent the LB to the end user. Here are the key elements:

1) Load Balancer - the container for the load balancer rules. Has properties:

• sourceIpAddress - the address to access the load balancer
• name - unique LB name
• algorithm

2) Load Balancer Listener - the listener inside the load balancer, listening for a specific source/instancePort. Has properties:

• sourcePort - Specifies the Load Balancer port
• instancePort - Specifies the TCP port on which the instance is listening

3) Load Balancer Instance 

VM instance registered inside the load balancer. Once the instance is a part of the load balancer, it starts receiving requests coming to the LB source Ip/Lb listener(s) source port(s):

New APIs

DB changes

TBD

How to list Guest IP addresses allocated for LB purpose

At the moment, cloudStack doesn't expose any API for listing IP addresses from the guest network. Adding a new one to serve this purpose:

In 4.2 this API will return only Allocated IP addresses.

Internal Load Balancing Vm management and life cycle

1) The InternalLBVM will be created with 2 interfaces: eth0 - linkLocal (private in VMWare case), eth1 - the IP address of the LB rule.

3) InternalLBVM will be managed by InternalLBNetworkApplianceManager

2) InternalLBVM life cycle:

• Create: InternalLBVM gets created when the first Load Balancing Rule is created for the IP address
• Destroy: InternalLBVM gets destroyed when the last Load Balancer is removed for the IP address
• Reboot: InternalLBVM can be rebooted as a regular system vm using RebootSystemVm API.
• List: InternalLBVM can be listed with ListSystemVMs API

Limitations

• Internal and Public Lb are mutually exclusive on a network/tier. If the tier has LB on the public side, then it can't have the Internal LB
• Supported just on VPC networks in 4.2 release.

UI

TBD