CPU and RAM Overcommit

Introduction:

This document describes the Cpu/Ram overcommit feature.

In the current implementation the cpu overcommit is global configuration value. This needs to be changed to provide a more granular control over the overcommit parameters. Currently there is no provision for ram overcommit.

This feature implements the ram overcommit and allows the ram and cpu overcommit ratios to be specified on a per cluster basis.

Use case:

change the vm density on all the hosts in a given cluster. This can be done by specifying the cpu and ram overcommit ratios.

- Each cluster (depending on the hypervisor platform, storage or h/w configuration) can handle a different number of VMs per host/cluster - trying to normalize them can be inefficient, as the ratio has to be setup for the lowest common denominator - hence, we are providing a finer granularity for better utilization of resource, irrespective of what the placement algorithm decides

- when combined with dedicated resources, it gets better - with dedicated resources, we may have the capability to tell account A will use cluster X. If this account is paying for "gold" quality of service, perhaps, those clusters would have a ratio of 1. If they are paying for "bronze" QoS, their cluster ratio could be 2. 

Design description:

Admin can give the cpu and ram overcommit ratios at the time of creating a cluster or update the values after creating.

Cloudstack will deploy the vms based the overcommit ratios. If the overcommit ratio of a particular cluster is updated, only the vms deployed hereafter will be deployed based on the updated overcommit ratios. The overcommit ratios will be stored in the cluster details table. Also whenever we add a host we will check of the host has the capabilities to perform the cpu and ram overcommiting. These capabilities will be stored in the db.

supported Hypervisors.

XenServer
KVM
VMware

APIs:

AddCluster Will be modified to include the cpu and ram overcommit values in the cluster parameters.

UpdateOvercommitRatio Updates the overcommit ratios of a cluster.

If value is not provided we default it to 1, meaning no overcommiting of resources by default.

DB changes:

There will be no changes to the db. We will add the cpu and ram overcommit ratios in the cluster_details table.

Caveats

What should the behavior be if admin changes the overcommit factor for a cluster that conflicts with the current situation. For example,lets assume Cluster X has an over commit factor of 1.5x for memory and the admin wants to change this to 1x - i.e no overcommit (or changes from 2x to 1.5x) - however, based on the "older" factor, CS might already have assigned more VMs - when the admin reduces the overcommit value 

1. if there is no conflict, there is no issue

2a. if there is a conflict (i.e. current allocation would conflict with the new value) - should we reject this change?

2b. or accept the change but not add more VMs anymore ( preferred method)

if we decrease the factor - currently we allow doing that (say change from 2X to 1X) . Lets say If the allocation is beyond the factor already (say 1.5 X) then what it means is no future allocation will be allowed and secondly the dashboard would start showing >100% allocated which might confuse the admin (in our example it would show 150%).  The admin would also start getting alerts for capacity being already exhausted. i.e. we should accept the new value and allocate only if the system has enough capacity to deploy more VMs based on the new overcommit ratios.

But, say the allocation done till now is still within the new factor (say 0.8X is allocated currently) then allocation would still be allowed and dashboard would show 80% allocated so in this case everything seems to be correct and we should allow admin changing the factor.

Check if host has capabilities of overcommiting.

Almost all the hosts have the capability to overcommit, and it is up to the admin to make sure of it. Even if the host is not configured properly, cloudstack will try to set the parameters assuming it has capability.

Capacity calculations.

The overcommit ratios are dynamically plugged into the capacity calculations. All the capacity calculations is done based on the overcommitted value of capacities. So if the overcommit ratios is decreased the used capacity may go beyond 100%.
Example:
Overcommit =2
capacity = 2GB
capacity after overcommit = 4GB.
Now if we deploy 3 VM of 1 GB each
used =3GB
free = 1GB
used % = 3/4 *100 = 75%
if the overcommit ratio is decreased to 1
used = 3GB
free = -1GB
used % = 3/2 *100 =150% (will generate alerts based on this.)

Alert generation.

All the alerts are generated based on the global threshold values. Will change this behavior once we have
thresholds per cluster.

Upgrade senario.

1.) Disable the overcommit feature in clusters existing before the upgrade. The admin should manually enable it.

Task Breakup

Discussions with community 3 days.
updating functional spec 1 days.
Coding 4 days.
testing 2 days.

Prerequisites to use cpu and ram overcommit.

The feature is dependent on the OS type ,Hypervisor capabilities, and some scripts.

1.) All VMs should have a ballon driver installed in them. The hypervisor communicates with the ballon driver to free up and make the memory available to a guest. So in case of
XenSever.
The ballon driver can be found as a part of xen pv or PVHVM drivers. The xen pvhvm drivers are included in upstream linux kernels 2.6.36+.

VMware.
In case of VMware the ballon deiver can be found as a part of the vmware tools. All the guests that are deployed in a overcommit cluster should have the vmware tools installed.

KVM
All the guest are required to support the virtio drivers. All the linux kernels>= 2.6.25 have them installed. Admin needs to activate it. It can be done by setting CONFIG_VIRTIO_BALLOON=y in the virtio configuration.

2.) Hypervisor capabilities.
The hypervisor should be capable of using the memory ballooning. Each of the hypervisor provides their own feature to support ballooning.

xenserver
The DMC (Dynamic Memory Control)capability of the hypervisor should be enabled. only xenserver Advanced and above versions have this feature. In case of xenserver we cannot support an overcommit factor greater than 4. This is because of a hypervisor constraint.

VMware
The memory ballooning is supported by default.

KVM
kvm dose not support automatic adjustment of the guest OS memory dynamically.

Behavior when overcommit is changed while there are VMs running in the cluster.
*In this case all the VMs prior to the overcomit change will continue to run with the same settings. Only the
newly deployed VMs will have the new setting based on the new overcommit value.
*On overcommit change we scale the used and reserved capacities based on the new overcommit values. This happens when
the capacity checker runs. The time interval between consequent runs can be changed by changing the global parameter capacity.check.period.

• The capacity as seen by the cloudstack will not be correct until the capacity recalculate happens.

As of now cloudstack dose not check for any perquisites. It is the admin's responsibility to provision accordingly.

Thing To do

• Trigger the capacity recalculate when the overcommit is changed. No need to wait until capacity checker runs.
• Create action event on overcommit change.
• Create action event when capacity recalculate is complete.