Versions Compared

Old Version 19

changes.mady.by.user Likitha Shetty

Saved on

compared with

New Version Current

changes.mady.by.user Boris Stoyanov

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Bug Reference

CLOUDSTACK-4757

Branch

Work is in progress in ova-multiple-disks branchthis PR  https://github.com/apache/cloudstack/pull/2146

Tested the PR that was submitted by Citrix. Made architectural changes, bug fixes, enhancements and feature additions for it to be useful.

Introduction

OVA is an archive or an envelope specification that can contain a complete VM including multiple disks and other files such as ISOs. And CloudStack templates in case of VMware are in OVA format.

Today in CloudStack, we only support template creation based on OVA files that contain a single disk. If an Admin/User registers a template using OVA that contains more than 1 disk and disk or iso files and launches an instance using this template, only the first disk is attached to the new instance and the rest of the disks are and iso's are ignored. 

Purpose

Purpose of The purpose of this document is to present functional specifications for improvements that are required to extent to extend the current OVA support to include multiple disks.

...

Datadisk Template - An abstraction for a DATA volume contained in an OVA

Iso Template - An abstraction for an ISO contained in an OVA

Goals

The current CloudStack behavior needs to be improved to support OVA files with multiple disksdisks and/or an ISO.

In case of an instance launched from a template that is an OVA containing more than 1 disk 

  • First disk , disk should be attached to the new instance as the ROOT disk
  • Volumes should be created for the additional disks and attached to the instance as additional DATA disks
  • Any ISO contained is mounted as CD-ROM and connected.

Scope

Scope of The scope of this proposal is limited to adding support for additional disks present in an OVA that disks or ISO present in a simple OVA that is registered in CloudStack. It is not in the scope to fully support an OVA and hence CloudStack will ignore any other information that is encapsulated in the OVA which may include information about the ISO or networks presentnetworks, multiple Virtual machines. The limitations on OVA are contained in the Limitations section.

Feature specifications

Currently in , in CloudStack, template abstraction is that of a VM template that contains a single disk. To represent the additional disks present in an OVA we will introduce the concept of Datadisk templates.

...

  • Used to create a data disk from the volume image that is present in the template
  • Stored in in vm_templates tables  tables along with the current CS templates
  • Used only when a user tries to deploy a VM from the corresponding VM template and never independently

Datadisk templates will provide flexibility to orchestrate, deploy and manage additional disks present in an OVA like any other Datadisk in CloudStack.

Similarly, ISO templates are introduced in Cloudstack.

Architecture

Work flow changes

Template registration

  • No change in user User experience during template registration
  • Once template is downloaded and installed in the secondary storage it will be post processed
  • Post processing will be handled by 2 new Agent Secondary Storage commands - GetDataDisksCommand and CreateDataDiskTemplateCommand
    • GetDataDisksCommand  (this used to runs on MS by mounting SS, as the vmware agent is running in MS context) **1
      • Will temporarily import the OVA into CloudStack to identify the ROOT disk and
      Template post processing will return information about each of the
      • additional data disks present (if any).
      Information about
      • If present, path and size of each of the data disks will
      include size of the data disk, path where the template has been installed and the device id
      • be returned
      • Identifying the ROOT disk (remaining disks will be considered as data disks)
        • OVA template custom property "cloud.ova.bootdisk" will be used to determine the ROOT disk. If this property is not set the first disk will be identified as the ROOT disk (again there is a check for this in the code, but surprisingly some non cloudstack ova seems to be working) The OVA template can come from non cloudstack sources so the custom cloudstack property check is removed, instead the right controller is configured thru cloudstack so that the resulting VM boots properly.
        • The value of parameter “cloud.ova.bootdisk” should be in format ”<controllertype><controllerId>:<devicenumber_on_this_controller>” e.g. “cloud.ova.bootdisk”=”scsi0:2”
          • Possible values for <controllertype> - "ide" or "scsi", <controllerId> - 0, 1, 2, 3, <devicenumber_on_this_controller> - 0 to 15
          • Custom property can be added to VM before creating OVA from the VM per VMware documentation. Refer -  Modifying Virtual machine custom property
      • For each of the additional disks
      a
      • , Datadisk and ISO template entries will be
      created
      • made in the DB
        • Entries will be made
        in
        • in vm_templates and vm_template_store_ref table
        • Datadisk Template will be referenced to the corresponding VM template by
        populating vm_template_details for VM template with key value pair - child_template_id,  <datadisk_template_id>
        • with DB column parent_template_id where parent_template_id points to the VM template
    • CreateDataDiskTemplateCommand
      • This agent command will be invoked for each of the templates returned by GetDataDisksCommand
      • As part of the Datadisk template creation following files will be created to represent the template
        • An OVF descriptor based on the disk properties returned by GetDataDisksCommand
        • The OVF descriptor will be cleaned up for each of the disk to carry information just for that disk.
        • An OVA .meta file to represent an OVA
        • template.properties file to encapsulate the properties of the Datadisk template
      • Datadisk, OVF, META and template.properties file will be present in a separate folder e.g. -
        • Say the Datadisk template id is 223 then it will be present in folder  - template/tmpl/2/223/
        • Sample contents of the Datadisk template folder will be
          • 223-2-9f6a1c42-472b-3bd1-9fac-1101a6228df9.ova.meta multipledisk.ovf multipledisk-disk1.vmdk template.properties
      • If the creation of any of the individual Datadisk templates fails then we will rollback the VM template and the associated data disks that were created.

 

Handling of ISO's

 

The other functionality currently missing in the import of OVA is the treatment of ISOs. The ISOs are simply ignored. The ISO files mentioned in the OVF will be mounted as CD-ROM that is to say that ISOs mentioned in the OVF should be backing up a CD-ROM device.

 

Listing templates

  • All template i.e. both VM templates and Datadisk templates will be listed.
  • They will Datadisk template can be differentiated from VM templates by the following attributes present in TemplateResponse – bootable, guestos and parenttemplateidproperty templatetypeFollowing attributes will also be different for the two type of templates – bootable(set to false for Datadisk template), guestos(unset for Datadisk template)
    • Datadisk templates will have templatetype set to 'Datadisk'
    • VM templates will continue to have templatetype set to 'User'

Virtual Machine deployment

  • If user tries to deploy a VM from a template containing multiple disks, user will have to choose a Datadisk offering for every Datadisk template chosen to be deployed along with the VM template
    • If the actual size of the disk exceeds the size of the disk offering then VM deployment will fail
    During templateDuring VM create process, when CloudStack makes DB entries entry for the ROOT disk, Volume DB entries will be made corresponding to in volumes table for each of the Datadisk templatetemplates
    • If the actual size of the disk exceeds the size of the disk offering then VM deployment will fail
    • The UI is modified to let the user select disk offerings for each contained disk.
    •  Volume state will be updated to Uploaded state because the volume is already present in secondary storage and has been packaged into an OVA during template post processing
  • During VM orchestration while the storage is being prepared, a volume will be created in primary storage for each of the additional disks
    • Each of the volumes will be migrated from secondary storage to the primary storage chosen by the deployment planner
    • When VM starts it will be booted with all the chosen disks and ISO

  • The disks are resized to match the disk offering selected for the disk. 

Template deletion

  • Deletion of DatadiskTemplates individual Datadisk Templates will be allowed
  • When a VM template is deleted, all the corresponding Datadisk templates will remaindeleted

Template copy

  • Copy template will not be supported on a individual Datadisk template
  • When a VM template is copied to a zone, all the corresponding Datadisk templates will also be copied
  • Copy template will not be supported on a individual Datadisk template
  • to the zone

Template update

  • Users will be allowed to update Datadisk templates
    • Template attributes that can be updated - bootable, displayext, format, name, ostypeid, sortkey

Update template permissions

  • Not supported for Datadisk Templates
  • By default the Datadisk templates will be private and non-featured

Change in the existing behavior

  • When a user downloads a CS template with multiple volumes, the entity that is downloaded will not be same as the one that was registered
    • The downloaded template will only contain the ROOT disk because when a OVA template is registered it is now consumed and split into 2 or more templates (VMTemplate VM Template and DatadsikTemplateDatadisk Template(s))

Web Service API

DeployVirtualMachine API

New parameter – disktemplatetodiskofferinglist– datadisktemplatetodiskofferinglist

  • Mapping from DiskTemplate Datadisk Template to DiskOfferingDatadisk Offering
  • If a VM is being deployed from an OVA template that contains additional disks, then during the deployment user will have to choose a Datadisk offering for every Datadisk template he/she chooses to deploy along with the VM template
  • Mutually exclusive with the existing parameter parameter diskofferingid
  • Optional
  • Usage - &disktemplatetodiskofferinglist[0].disktemplateid=2&disktemplatetodiskofferinglist[0].diskofferingid=3

...

  • While listing all templates list only VM templates i.e. non Datadisk templates (VM templates will have bootable property set to true)Once templatesOnce the user has selected the VM template allow user to pick between disk offering or Datadisk template with the help of radio buttons (user can choose to select neither)

    • VM templates can be distinguished from the Data disk templates by property ‘templatetype’. VM templates will have the type set to USER whereas Datadisk templates will have the type set to DATADISK.

  • If user chooses disk offering then rest of the work flow will remain the same as today
  • If user chooses Datadisk template then list all the Datadisk templates that belong to the VM template chosen in the first step (templates that have parentTemplateId parameter set to that of VM template)
DB Schema changes

    ...

    • Add new column parent_template_id to vm_templates table

      • default value is null

      • for Datadisk templates, value will be Id of the VM template that the Datadisk template belongs to

      • for VM templates, value will always be null

    Upgrade scenario

    • In a setup that has been upgraded to 4.45, templates with multiple volumes
      • That were registered prior to upgrade will continue to operate as before. Any VM launched from these templates will only have the ROOT disk
      • That will be registered after upgrade will follow the new behavior. Any VM launched from these templates will have additional data disks attached to them along with the ROOT disk (i.e. if user chooses to)

    ...

    • CloudStack templates and uploaded volumes in case of VMware are in OVA format. For 4.4 10 release, we will be adding multiple disk OVA support only for CloudStack templates and not uploaded volumes.
    • In 4.410, users will not be allowed to perform any independent operations on the Datadisk templates
      • User cannot use the Datadisk template to add a data disk volume after VM is created based on the template
      • Following operations will not be supported on the Datadisk templates - updateTemplatePermissions, extractTemplate, prepareTemplate (Future work)
    • While deploying a Virtual Machine, user will not be able to provide custom disk offerings for datadisk templatesfor datadisk templates
      • disktemplatetodiskofferinglist can only have disk offerings that are non-custom

    Limitations placed by Cloudstack

    Following are the limitations that are placed by current Cloudstack implementation and its limitation.

    1. Cloudstack allows single disk controller for all data disk. In case the OVA contains multiple data disk only one type of disk controller will be applicable for all of them.
    2. Cloudstack allows iscsi disks to be resized. Due to this limitation, if datadisk are ide for example, the resize to disk offering size will fail. And user will see actual disk sizes contained in OVA in the vCenter, not the ones selected by disk offerings.
    3. The disk controller specified in the OVA file could have a wide range of types and sub-types. Cloudstack has only limited types available. In case the disk controller is not supported by cloudstack, there can be boot issues with the VM. 
    4. If ISO is attached from CloudStack, it's not being picked up when exporting the VM. Therefore, in order to prepare proper template containing ISO, it must be attached from vSphere, connected, with the option to 'connect when power on' before being shut down.  

    Limitations on OVA

    There are some limitations on OVA template that Cloudstack will be able to handle. These limitations if found restricting can be further worked upon and removed.

    Some assumptions on OVF files contained in the OVA packages are:

    • The first disk is considered to be the boot disk, if it is an iso instead the VM creation will fail. If not it can result in boot issues.
    • All standard parmeters like size should be in there for all the vmdks and ISOs contained in OVA
    • An ovf file is considered to be in proper format and with valid information.
    • It is also assumed that the imported OVA file is compatible with vCenter
    • Multi-disk OVA file that bundles a single VM with multiple disks and an optional ISO
    • The disks should be bundled in the OVA with only relative paths in the OVF file (href attribute)
    • The disk offering size utilized for data disks needs to be equal or larger to original disk size in the OVA
    • There will not be any empty disk in the OVF file, all disks will point to a valid File resource
    • Since Cloudstack treats each disk and iso file as separate unit, the specifications contained in InstallSection or StartupSections of OVF are ignored.
    • Cloudstack has its own mapping for operating systems and the information contained in OperatingSystemSection of OVF is ignored
    • The files are in uncompressed format, 'ovf:compression' attribute on File will be ignored
    • Files are not chunked and 'ovf:chunkSize' attribute is ignored
    • Any manifest and certificate files will be ignored
    • Disk format vmdk for data disk
    • Essentially we will only be looking at disk information in the ovf and other spec will be ignored
    • If there is a ISO in the ovf package then it will be mounted as CD-ROM
    • The network section is ignored and network is instrumented by Cloudstack.
    • There's no error handling in the UI when copying additional disks and ISOs, it has been observed that some templates appears as ready but are not deployable, due to a copy failure. The admin needs to monitor logs and check the template on storage when installed.