Introduction

CloudStack has a significant advantage in its ability to support multiple hypervisor types within the same zone. CloudStack implements a component model in which “plugins” can be loaded for each kind of hypervisor supported. These plugins can be implemented independently, and there exist plugins for Xen, VMWare, OVM, but not Hyper-V.

Purpose

This functional specification describes the requirements for supporting the Hyper-V hypervisor. Hyper-V support goes beyond mere hypervisor control. The specification allows for storage models typical of a Hyper-V deployment, and it notes required changes to the system VMs.

References

Docs:

Hyper-V Server 2012 licensing http://blogs.technet.com/b/keithmayer/archive/2012/09/07/getting-started-with-hyper-v-server-2012-hyperv-virtualization-itpro.aspx#.UJbNRGeYJUk
Hyper-V SMB 3.0 support http://blogs.technet.com/b/josebda/archive/2012/08/26/updated-links-on-windows-server-2012-file-server-and-smb-3-0.aspx
Hyper-V’s WMI V2 API http://msdn.microsoft.com/en-us/library/hh850319%28v=vs.85%29.aspx
Hyper-V’s PowerShell API http://technet.microsoft.com/en-us/library/hh848559.aspx
Hyper-V’s supported Guest OS’ http://technet.microsoft.com/en-GB/library/hh831531.aspx
Anatomy of a CloudStack Plugin http://www.slideshare.net/gavin_lee/cloud-stack-overview/60
Overview of CloudStack development https://cwiki.apache.org/CLOUDSTACK/index.html#Index-

Projects:

Document History

V0.1 – 2012-11-04

Glossary

See Architecture and Design description.
See also Feature Specifications.

Feature Specifications

A hypervisor plugin to control Hyper-V Server 2012 (V3.0) must be produced to support the full range of CloudStack hypervisor operations. A plugin able to control Hyper-V Server 2012 can also control a Windows Server 2012 with the Hyper-V role activated, as Hyper-V Server 2012 is a subset of Windows Server 2012 functionality. Hyper-V is controlled via a WMI API or PowerShell scripting. PowerShell scripts are built on WMI API calls. Neither WMI nor PowerShell script execution is addressed by existing plugins. Therefore, the focus of this feature is a new hypervisor plugin that can translate CloudStack operations to WMI and/or PowerShell.

Hyper-V support will include the ability to use SMB shares for primary storage and what is currently referred to as secondary storage. To coincide with Hyper-V 2012 (V3.0), Microsoft upgraded SMB to provide robust support for hosting VM disk images. Now the virtual disk for a Hyper-V 3.0 VM can sit on a SMB 3.0 share. Therefore, a new storage type will be added to CloudStack.

Hyper-V Server 2012 does not include a NFS-client feature. Therefore, the NFS option will not be supported for primary storage on this hypervisor type.

The existing set of networking offerings will be available. The virtual router system VM can run on a Hyper-V host, albeit using a different Linux distribution than the current system VM. This VM delivers the advanced networking services. The security group feature of basic networking will be implemented on the Hyper-V host. Therefore, Hyper-V should not introduce any limits to CloudStack's networking.

A console proxy system VM is required than can deal with RDP. Hyper-V provides access to each VM’s frame buffer via an RDP server. For example, the console tool vmconnect.exe can access the console of a Linux VM running on a remote Hyper-V even when that VM lacks Linux Integration Services. As with the virtual router, the console proxy VM has to run a version of Linux supported by HyperV. Therefore, a new console proxy system VM is required.

The existing user experience is left unchanged. Besides retaining the existing hypervisor feature set, Hyper-V will be configured using the existing workflows. The CloudStack GUI will have only minor updates to allow it to distinguish settings specific to Hyper-V. For example, a VHDX disk type will have to be added along with a SMB primary storage type. Therefore, no changes to the GUI or its wizards will be required.

Architecture and Design description

This spec relies on the CloudStack plugin model, the agent bus, and knowledge of the upcoming “Javelin” storage architecture to make implementing Hyper-V support manageable. The plugin model breaks down the implementation task into well encapsulated pieces. The Message Bus makes remote access to the Hyper-V seamless to plugins, which allows non-functional requirements to be deferred and possibly avoided. “Javelin” storage removes limits imposed by the existing secondary storage model. These architectural elements allow basic requirements to be implemented starting immediately while a development model for system VMs is worked out.

Only architecture that makes adding hypervisor support manageable is of interest. The hypervisor support should be divided from the overall CloudStack such that development and testing can be independent of the CloudStack management server. Non-functional details associated with deployment and maintenance of hypervisor control code should be minimized or avoided altogether. Finally, any future simplifications of the current design should be adopted. Therefore, this section only mentions architecture known to simplify feature development.

The plugin model allows independent development of the feature by dividing the device-specific code from the management-server. First, the plugins are compiled independently and loaded into CloudStack according to a configuration file. The plugin need only implement desired plugin API interfaces. The details of the management server are hidden. Secondly, the plugins themselves encapsulate device-specific code in a ServerResource. These divisions are visible in the figure “Anatomy of a Plugin” below. The divisions within a plugin allow the development effort to concentrate on creating a ServerResource suitable for Hyper-V. The remainder of the Hyper-V plugin would reuse code from an existing hypervisor plugin. Therefore, there must be a plugin API in place for each area we wish to change.

Message Bus binds the two halves of a plugin together. ‘Message Bus’ refers to a basket of activities involved in setting up and carrying out communication between ServerResources and their Plugin counter parts in the CloudStack management server. The Message Bus includes a protocol for establishing communications with the management server, management of the resulting TCP connection, and an agreed network format for transmitting request and response objects. The bus’ specification is embodied by its implementation, which is in two parts. In the management server is an Agent Manager that listens for connections and transmits commands objects on behalf of Plugins. At the receiving end is an Agent, currently written in Java. The Agent loads a ServerResource, initiates connection to the management server, and routes requests and responses to and from the ServerResource. Therefore, the Message Bus provides a logical abstraction for communications between the two halves of a Plugin.

This Message Bus allows us to defer decisions on non-functional requirements. Non-functional requirements arise in conjunction with local deployment of a ServerResource. In this case, ServerResource installation, upgrade support and native implementation of an agent have to be considered. Fortunately, whether an Agent executes on the management server itself or on host in the data center is of no consequence to the ServerResource. The handshaking carried out by the Agent will be the same in both cases. The Hyper-V host can be controlled remotely via the WS-Management SOAP interface, or the Hyper-V host can be controlled via local WMI or PowerShell calls as show in the “Hypervisor Control: Direct or Agent” below. Therefore, the Message Bus’ design allows non-functional requirements to be avoided in at least the initial phases of plugin implementation.

To avoid the limits of Secondary Storage, SMB3.0 support for template and snapshot storage should be delayed until project Javelin is available. The HypervisorGuru API used for hypervisor control does not include operations for secondary storage control. Nor is secondary storage modeled with another Plugin API. Instead, the server types supported by Secondary Storage are tightly coupled to the CloudStack kernel code. However, a more flexible approach to write once, read many (WORM) storage will be introduced with project Javelin. This model introduces new abstractions to decouple image transfer from specific file sharing protocols such as NFS. It will also make manipulate WORM storage via plugins. Therefore, secondary storage will change too significantly to be properly discussed at this point.

This architecture allows CloudStack commands to be implemented starting immediately. The Message Bus agent implements the infrastructure for starting and passing messages to a ServerResource. The implementation work for a ServerResource can concentrate on command objects immediately and independently of the management-server. The management-server side of a plugin can use an existing plugin as a starting point, as the device-specific details are largely contained in the ServerResource. Where system VM services are required for management server operation, these can be obtained by surrogate. In this case, the surrogate is a cluster of an already supported hypervisor type. This cluster will execute the required system VM. Therefore, the architecture takes care of provising the infrastructure that would otherwise delay work on basic features.

The ability to create plugins independent of the overall system buys time while the programming model for system VMs is determined. Splitting the system VM from its O/S and providing alternative console proxy VMs needs to be better addressed in the CloudStack architecture documentation. Alternatively, or a project to make system VM modification clearer should be undertaken. In either case, the missing architectural details are not an impediment to plugin development.

To summarise, the plugin model allows Hyper-V support to be broken into independently implemented pieces. The Message Bus provides sufficient infrastructure to allow development to concentrate on hypervisor commands and not infrastructure. As a result, work on basic features can begin before the rigid secondary storage model is replaced by the “Javelin” architecture. Likewise, a programming model for system VM features can be created while plugin development is carried on.