Page History

This is an early draft

Bug Reference

None

Branch

Work in progress patches are here:

https://github.com/djs55/cloudstack/tree/virsh-capabilities

Introduction

Some people prefer to manage their hypervisors entirely via the libvirt control plane. At the moment these people are forced to choose the KVM hypervisor, even though libvirt also supports the Xen hypervisor. This design extends CloudStack to support management of hosts running Xen via libvirt as well as KVM. No change will be made to the XenServer plugin.

Bug Reference

None

Branch

Work in progress (ugly) patches are here:

https://github.com/djs55/cloudstack/tree/virsh-capabilities

Principles

1. Maximise code shared between KVM and Xen. Corollary: minimise the number of Xen-specific workarounds
   1. We want people to be able to add features to the libvirt plugin without having to write unnecessary special-cases for Xen or KVM.
   2. If a libvirt hypervisor driver lacks a useful feature, we should implement it.
   3. Share the same system VM template as KVM

...

• this section is intentionally left blank

Document History

• 2014-06-08: initial draft

Glossary

Feature Specifications

...

• L1: It will not be possible to support clusters of mixed KVM/Xen hosts. This is because it is not possible to live migrate VMs between different hypervisors.
• L2: It will not be possible to support clusters of mixed Xen/XenServer hosts.
• L3: It will not be possible to automatically convert a XenServer cluster into a Xen cluster or vice-versa

Proposed changes

• C1: The HypervisorType enum will be extended to include "XEN" (alongside "KVM" and "XENSERVER")
• C2: Create a "XenLibvirtDiscoverer" as a subclass of LibvirtServerDiscoverer (alongside the KVM and LXC versions).
• C3: Generalise the cloudutils python function "isKVMEnabled" to "isHypervisorEnabled"
• C3.1 Detect the Xen hypervisor by reading /sys/hypervisor/type
• C3.2 Detect the KVM hypervisor by "lsmod | grep kvm"
• C4. Register a system VM template for the "XEN" hypervisor
• C4.1 Use the same template image as KVM (including keeping it in .qcow2 format)
• C5. Use the "xen" PV driver family if using "XEN"; and use "virtio" drivers if using "KVM"
• C6. Under Xen, by default boot the system VM in PV mode
• C6.1 Add a config setting to enable system VMs to boot in HVM mode
• C7 In cloud-early-config under Xen, first check the host private "channel" and fall back to the command-line parsing

...

• D1 Xen's libxl needs to support  

• put a summary or a brief description of the feature in question 
• list what is deliberately not supported or what the feature will not offer - to clear any prospective ambiguities
• list all open items or unresolved issues the developer is unable to decide about without further discussion
• quality risks (test guidelines)
  • functional
  • non functional: performance, scalability, stability, overload scenarios, etc
  • corner cases and boundary conditions
  • negative usage scenarios
• specify supportability characteristics:
  • what new logging (or at least the important one) is introduced
  • how to debug and troubleshoot
  • what are the audit events 
  • list JMX interfaces
  • graceful failure and recovery scenarios
  • possible fallback or work around route if feature does not work as expected, if those workarounds do exist ofcourse.
  • if feature depends other run-time environment related requirements, provide sanity check list for support people to run
• explain configuration characteristics:
  • configuration parameters or files introduced/changed
  • branding parameters or files introduced/changed
  • highlight parameters for performance tweaking
  • highlight how installation/upgrade scenarios change
• deployment requirements (fresh install vs. upgrade) if any
• system requirements: memory, CPU, desk space, etc
• interoperability and compatibility requirements:
  • OS
  • xenserver, hypervisors
  • storage, networks, other
• list localization and internationalization specifications 
• explain the impact and possible upgrade/migration solution introduced by the feature 
• explain performance & scalability implications when feature is used from small scale to large scale
• explain security specifications
  • list your evaluation of possible security attacks against the feature and the answers in your design* *
• explain marketing specifications
• explain levels or types of users communities of this feature (e.g. admin, user, etc)

Use cases

put the relevant use case/stories to explain how the feature is going to be used/work

Architecture and Design description

• discussion of alternatives amongst design ideas, their resources/time tradeoffs and limitations. Explain why a certain design idea is chosen over others
• highlight architectural patterns being used (queues, async/sync, state machines, etc)
• talk about main algorithms used
• explain what components are being changed and what the dependent components are
• regarding database: talk about tables being added/modified
• performance implications: what are the improvements or risks introduced to capacity, response time, resources usage and other relevant KPIs
• preferably show class diagrams, sequence diagrams and state diagrams
• if possible, publish signatures of all methods classes and interfaces implement, and the explain the object information of different classes

Web Services APIs

list changes to existing web services APIs and new APIs introduced with signatures and throughout documentation

UI flow

• either demonstrate it visually here or link to relevant mockups

IP Clearance

• what dependencies will you be adding to the project?
• are you expecting to include any code developed outside the Apache CloudStack project?

Appendix

Appendix A:

• a bidirectional host <-> guest private channel
• D2 libvirt's libxl driver needs to map <channel> in the domain xml to libxl constructs

Todo: document minimum Xen, libvirt versions

Risks

• The libxl driver in libvirt is less mature than the KVM driver. This might manifest as quality issues or missing features

Use cases

A cloud admin wishes to manage hosts with the libvirt control-plane (e.g. virt-manager, virsh etc) using the Xen hypervisor. The admin installs Xen and libvirt via their favourite Linux distribution. The admin logs into the CloudStack UI and creates a cluster with hypervisor type "XEN". The admin then adds a host to the cluster. A user is requests an instance and a VM is spawned on the Xen host via libvirt.

Architecture and Design description

On the choice of system VM image format: there is no need to choose a different format for Xen vs KVM as they are both able to use the same disk drivers via qemu. Therefore the Xen system VM image format should be set to .qcow2 to minimise divergence.

On using the host <-> guest private <channel> for configuration: this mechanism is used by KVM today and is a sensible design choice and a real missing feature in the libvirt libxl driver. Therefore we should implement the missing feature.

On extending the HypervisorType enum: this is a bit ugly because conceptually we have a single "libvirt" type with 2 subtypes for "Xen" and "KVM". All the code which currently says "if kvm`' needs to become "if kvm || xen".

On detecting hypervisors on the host: it would be better to always use /sys/hypervisor/type, but this doesn't appear to be populated by the KVM module. Another option is to query "virsh capabilities" but this requires the libvirt daemon to be already started, which is not required today.

Web Services APIs

No new APIs are needed

UI flow

No change to the UI flow is needed

IP Clearance

No new dependencies need to be added to CloudStackAppendix B: