Page History

...

GPU group - a group has one of more cards of the same type.

Introduction

GPU (graphics processing unit) is used for accelerating the image output in a frame buffer intended for output to a display. GPU-accelerated computing offers unprecedented application performance by offloading compute-intensive portions of the application to the GPU, while the remainder of the code still runs on the CPU. From a user's perspective, applications simply run significantly faster ^[1]. GPU can also be used as General Purpose Computing as GPGPU(General-purpose computing on graphics processing units) which has significant importance in the field of machine learning, image processing, linear algebra, statistics and Cloud Computing.

Now a daysNowadays, physical servers are coming up with graphic cards that have multiple GPUs, VMs running on the cloud can leverage the high computation power of GPU to meet high graphics processing requirements like Auto-CAD, Photoshop etc. Also, there are cards in the market which supports support sharing of a GPU cards card among multiple VMs by creating vGPUs for each VM, e.g. NVIDIA has introduced vGPU capable cards GRID K1 and K2 which allow multiple vGPUs on a single physical GPU.

With vGPU technology, the graphics commands of each virtual machine are passed directly to the underlying dedicated GPU, without translation by the hypervisor. This allows the GPU hardware to be time-sliced and shared across multiple VMs ^[2].

Xenserver XenServer has added the support for NVIDIA GRID K1 and GRID K2 cards ^[3]. It allows the VMs on XenServer hosts to use the GPU cards in following ways:

• GPU-passthrough: It allows the hypervisor to assign the entire PGPU to a VM, this is useful for a power users.
• VGPU: It allows the VM to share a PGPU device with other VMs, this is useful for tier 2 users.

...

This is the functional specification of the feature "GPU and vGPU support for CloudStack Guest VMs" which has jira JIRA ids CS-4760 and CS 4762.

Bug Reference

...

• CloudStack does following:
  • Add hosts with GPU/vGPU capability provisioned by admin.
  • Query hypervisor to obtain information about the cards, supported vGPU types (in the case of grid cards), the capacity of the cards.
  • Create compute offering with GPU/vGPU capability.
  • Deploy VM with GPU/vGPU support.
  • Destroy VM with GPU/vGPU support and releases the resources accordingly.
  • Allow the user to add GPU/vGPU support to a VM (without GPU/vGPU support) by changing service offering and vice-versa.
  • Migrate VMs (cold migration) with GPU/vGPU support.
  • Manages GPU cards capacity.
• Gracefully failure and recovery scenario.
• CloudStack generates alerts whenever a vm VM deployment with GPU/vGPU service offering is requested but there are not suitable GPU/vGPU capacity available.
• CloudStack generates suitable usage events when a vm VM is deployed, destroyed or migrated using GPU/vGPU.
• Global configuration:
  • None
• In XenserverXenServer, there are two algorithm algorithms to create vGPUs in the given set of PGPUs:
  • • DFS
    • BFS
  • Default allocation algorithm is DFS.

...

1. Add a host which contains GPU card:
   1. Admin adds a host with existing addHost API command and cloudstack CloudStack checks if the host is GPU enabled or not.
   2. Cloudstack queries the hypervisor and detect detects if the given host is GPU enabled.
   3. If the host has GPU card in it, then CloudStack queries the hypervisor and as a response, it gets the GPU group information, which get stores gets stored in the host_gpu_groups table.
   4. If the cards has have only GPU-passthrough support, then CloudStack stores the vGPU type as passthrough in the vgpu_types table.
   5. If the cards also has have vGPU support, then CloudStack checks for the enabled vGPU types in the hypervisor and stores them in the vgpu_types table.
   6. CloudStack also manages these cards by maintaining the capacity of each cardscard.
2. Create a compute offering with GPU/vGPU support:
   1. Create a compute offering with following two options:
      1. GPU-passthrough: To assign entire PGPU.
      2. vGPU: To share a PGPU device. Admin also needs to pass the vGPU type. Steps are explained in Web Services APIs section.
3. Deploy a VM with GPU/vGPU support:
   1. Admin/user can deploy a VM with GPU/vGPU support by selecting appropriate service offering.
   2. Once the deployment of VM with GPU/vGPU support is triggered, the allocator will decide which host to choose for VM deployment based on following criteria:
      1. Host The host has GPU cards in it.
      2. In case of vGPU, it check checks if cards has have the required vGPU type support.
      3. Cards has have enough capacity available.
   3. Once the host is final, Cloudstack will go ahead with VM deployment.
   4. In case if allocator doesn't finds find any appropriate host, then it will through an InsufficientServerCapacity exception.
   5. Once the VM deployment is successful, CloudStack will make a resource call to the hypervisor to get the remaining GPU capacity in the card and update the DB accordingly.
4. Deploy a VM without GPU/vGPU support (This scenario is to protect GPU resources and deploy this VM in non-GPU hosts):
   1. Admin/user can deploy a VM without GPU/vGPU support by selecting non-GPU service offering.
   2. Once the deployment of VM without GPU/vGPU is triggered, the planner and allocator will decide which cluster and which host to choose for VM deployment based on following criteria:
      1. Once the planner gets a list of clusters which has enough capacity to host this VM, deployment planner reorders this cluster list to put clusters with GPU enabled hosts at the bottom of the list. Please look at next section for more details.
      2. When planner passes this the list of clusters to the allocator, then allocator checks all the hosts in that cluster and if there is a GPU-enabled host in that cluster, allocator reorders this host list and put GPU enabled hosts at the bottom of the list.
   3. Once the host is finalized, CloudStack will continue with VM deployment.
   4. In case if allocator doesn't finds any appropriate host, then it will through an InsufficientServerCapacity exception.
5. Destroy a VM with GPU/vGPU support:
   1. Admin/user can destroy a VM with GPU/vGPU support by navigating through the existing destroyVM flow.
   2. Hypervisor will detach the GPU card from the VM and make it available to use.
   3. Once the destroy VM is successful, CloudStack will make a resource call to the hypervisor to get the remaining GPU capacity in the card and update the DB accordingly.
6. Migrate(cold migration) a VM:
   1. Search for hosts available for VM migration and satisfies GPU requirement.
   2. If the host is available, stop the VM in the current host and perform the VM migration task, else
   3. If the VM migration is successful, update the remaining GPU capacity for both the hosts accordingly.

...

Default first fit planner only look at the CPU and memory capacity of the cluster to create an ordered cluster list. The (host) allocator then goes over the hosts in a cluster to find a suitable host for deployment. If a suitable host isn't found, the next cluster in the ordered list is searched for a deployment destination. Since the default planner isn't aware of the GPU capabilities of a host, it may lead to a situation where vms VMS not requiring GPU capabilities use up cpuCPU/memory resources on GPU capable host. To address this issue the cluster list needs to ordered order such that for non-GPU vmsVMs, clusters with GPU hosts are towards the end of the ordered list. The solution needs to be generic enough to handle use cases for other capabilities in future.

...

• A concept of implicit tags for hosts has been introduced. All hosts with GPU capabilities will be tagged with an implicit tag ‘GPU’. The default planner will order the cluster list from least -> most count of unique implicit tags within the cluster.

• Implicit tags will be implemented as a comma separated list of global configuration parameters. Category The category will be ‘Hidden’. Initially, ‘GPU’ will be the only implicit tag available. The list can grow as support for other features is added in future.

• When management server connects to a host it’ll tag a host with implicit tag ‘GPU’, if the host has GPU capabilities.

• In addition to the cpu CPU and memory heuristics, the default planner will order the clusters from least to -> most count of unique implicit tags in a cluster.

• This will make sure clusters which do not have GPU hosts in them are picked first for deploying non-gpu GPU instances.

DB Schema changes:

...

• vgpu_types (id, vgpu_type, gpu_group_id, remaining_vm_capacity) :  This table contains the record of available vgpu profiles.

  Field	Type	NULL	Key	Default	Comments
  id	bigint(20) unsigned	NO	PRI	NULL	auto_increment
  gpu_group_id	bigint(20) unsigned	NO		NULL	"id" in the host_gpu_groups table.
  vgpu_type	varchar(40)	NO		NULL	Name label of the vgpu type.
  video_ram	bigint(20) unsigned			NULL	video RAM for this vgpu type.
  max_heads	bigint(20) unsigned			NULL	maximum displays per user.
  max_resolution_x	bigint(20) unsigned			NULL	maximum X resolution per display.
  max_resolution_y	bigint(20) unsigned			NULL	maximum Y resolution per display.
  max_gpu_per_pgpu	bigint(20) unsigned			NULL	maximum no. of vgpu per gpu (pgpu).
  remaining_capacity	bigint(20) unsigned			NULL	remaining vgpu vGPU can be created with this vgpu type on the given gpu group.
  max_capacity	bigint(20) unsigned			NULL	maximum vgpu can be created with this vgpu type on the given gpu group.

...