Bug Reference

The Jira issue associated with this design spec

https://issues.apache.org/jira/browse/CLOUDSTACK-664 https://issues.apache.org/jira/browse/CLOUDSTACK-665

Branch

What branch is this work being done in Master

Introduction

Currently users can create LB rule and configure stickness policy, Algorithm, Autoscale etc.This feature will enable the users to configure Health check on the LB rule.

Purpose

1. To ensure if the backend services are healthy
2. Ensure the loadbalancing rule dosen't send any furthur requests to an unhealthy backend service(s).

Benefit of configuring Health Checks on LB Rule:

With the ability to perform health checks on the right set of parameters, the load balancer can service all of the requests by forwarding requests to healthy VM's in case of a VM failure .The load balancer would exactly know, based on the health check configured, that the service is unavailable as supposed to the VM being unavailable.

Dynamic addition / removal of VMs: Once the health checks are configured, the load balancer can automatically remove a VM from load balancer pool if the health checks fail. Also, once the health checks are successful again, the load balancer can add the VMs back to the pool.

This is functional specifications for the feature https://issues.apache.org/jira/browse/CLOUDSTACK-443
f
State the purpose of the document; something like: this is functional specificationS of feature "..." which has Jira ID CS-xyzw

References

Configuring health checks in Aws http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/ConfigureHealthCheck.html
API to configure health check http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/APIReference/API_ConfigureHealthCheck.html

Document History

Feature Specifications

This feature will enable the users to configure health checks on the LB rule.

Limitations and Assumptions

The initial integration of this feature is based on the following limitations and assumptions:

• This feature will be applicable if NS device is present in the CS.
• Already existing health checks will not be managed. only CS created health checks will be managed.
• Only one health check will be configured on the LB rule.
• The protocol of LB rule will be used implicitly for configuring the protocol for health checks.
• ping path configurations of health check will be applicable only if the stickiness policy is LBCookie or Appcookie
• port configuration of health check will be used from the private port configuration on LB rule.
• Feature needs to be supported on NetScaler Load Balancers as well as CS VR.

Quality Risks/Test Guidelines

Functional

1. When a health check is created. A new health check will be created on the LB rule to all the services assigned to the LB rule.
2. When a health check is modified. Old health check will be delete and a new health check will be created.
3. When a health check is deleted, health check created on the NS will be deleted.
4. Modifying the lbrule_health check_time_interval global config, the health check sync time will be affected

   Scalability

5. currently only one thread will be running sync the health check status of all the LB rules. This might impact the performance/lag in syncing the state from NS to cloud DB.

Supportability characteristics

Logging

The HealthCheck manager process logs all the success of updating status of services to Cloud DB operations to DEBUG, all exceptions/failures to ERROR. It also provides DEBUG logging of connection parameters, and LB rule/ health check key information for all HealthCheck manager operations in DEBUG.

Debugging/Monitoring

In addition to logs, users can verify the healthchecks configured on the LB rules in NetScaler.

Configuration

A new configuration parameter will be added "lbrule.healthcheck.time.interval" default to 60sec. This parameter value is used to update the health check status from NS to Cloud DB

Branding

This enhancement has no impact on branding.

Upgrade/Migration

No upgrade scenarios need to be handled, as this is a new functionality.

Use cases

Users should be able to define health check policies as part of configuring the load balancer rules
User should be able to delete/modify health check policies as part of configuring the load balancer rules
Refer to AWS Health Check Configuration details for additional information and the screenshot below.

Architecture and Design description

Glossary

Service : An App running in a VM, which is assigned to a LB rule.
Response_Time : Time to wait when receiving a response from the health check (2sec - 60 sec), Default value is 5sec
Healthcheck_interval : Amount of time between health checks (0.1 min - 5min). Default value is 5sec
UnHealthy_thresshold : Number of consecutive health check failures before declaring an instance unhealthy.Default value is 10
Healthy_thresshold : Number of consecutive health check success before declaring an instance healthy. Default value is 2

Using NS API/ haproxy commands to create/delete/modify the health check monitor

Health Check Operations Supported in the feature :

Adding Health Check:

By using the createLBHealthCheck api command we can create the healtcheck monitor for the services in the Rule.When new VM’s got assigned to the rule, monitor should be added to these services also.

Modifying Health Check:

This is a batch operation. To modify the health check, first call deleteLBHealthCheck which will delete the already existing health check and call createLBHealthCheck with the new configuration parameters.
Note: if the new configuration parameters were not able to be applied, then it won’t restore the old health check.

Deleting Health Check:

To delete the healthcheck/monitor, call the deleteLBHealthCheck api which will delete health check from the LB rule.

Listing Health Checks:

Using the listLBHealthCheck, health check created on the LB rule will be retrieved.

Tables going to be added/modified:

Table name : loadbalancer_healthchecks

Web Services APIs

The following APIs are supported in this feature.

1. createLBHealthCheck Creates a Load Balancer Health Check
2. deleteLBHealthCheck Deletes a LB Health Check.
3. listLBHealthCheck Lists LB Health check

NetScaler Nitro API's classes :

lbmonitor
service
service_lbmonitor_binding

Sample Code to add monitor :

lbmonitor mymon = new lbmonitor();
mymon.set_monitorname("cshttpmon");
mymon.set_type(lbmonitor.typeEnum.HTTP);
lbmonitor.add(ns_session, mymon);

HAProxy.

TBD

UI flow

1. A new dialog will be shown to when clicked on the above button, so that user can provide the health check params
2. if the health check is created then status of the health check will be shown next to the VM in details page of LB rule.

Appendix

Appendix A:

Appendix B: