Background: When a data disk is needed by a VM that resides in a different cluster (i.e. different primary storage) than the current cluster/primary storage, it involves a costly/time consuming data copy operation - which involved copying data from one primary storage to another primary storage via the secondary storage. A primary storage is not shared across multiple clusters. As an example, AWS EBS provides the following benefits: EBS volumes are placed in a specific Availability Zone, and can then be attached to any instance in that same Availability Zone. Even though a volume can only be attached to one instance at a time, if an instance fails or is detached from an EBS volume, the volume can be attached to any other instance in that Availability Zone.

Requirement: CloudStack to support zone-wide (primary) block storage

- Multiple clusters can share the same primary storage (as long as the underlying hypervisor platform supports it)

- Only one VM can mount/attach a disk at any time (i.e. multiple VMs cannot attach the same disk concurrently)

- snapshot/restore features are similar to current semantics i.e. no degradation in features

- we should allow multiple zone-wide primary storages

-A cluster can have a cluster-specific and zone wide primary storage concurrently

- There is no requirement to support multiple hypervisors (XS/VMware for example) per zone. But zone wide primary storage can operate within boundary of specific hypervisor type that supports zone wide primary storage, E.g. KVM, VMware. Hence we can have zone with multiple hypervisors and that zone can have zone wide primary storage service one particular type of hypervisor which is chosen while creating the zone wide primary storage.

- upon migration, existing customers with more than 1 cluster per zone must not have any impact – any upgrade process needs to be clearly documented

- During the upgrade, no need to allow promotion of one primary storage to become the zone-wide primary storage;

- Adding zone-wide primary to existing zone will need to be possible

-          In addition to each current primary storage

-          In lieu of current primary – in this case, storage must be migrated manually, prior to deleting the existing primary storage

- Disk offering setting will now have choices for the admin on where the disk will be created – local, cluster-wide or zone-widestorage

 - How to migrate each primary storage data to “new” zone-wide – how can this be less painful - need to be clarified

Design

In the new storage framework, each storage can have its own life cycle, the storage will be not tied to a cluster or a host. 

The DB changes:

1. new column, called scope, will be added into storage_pool table. The scope has following attributes:

    HOST, CLUSTER, ZONE;

Host means the storage is attached to a particular hypervisor host, a.k.a, it's the local storage.

Cluster, means the storage is attached to a cluster, it's the primary storage we have today.

Zone, means the storage is attached to a zone, 

2. new column, called hypervisor, will be added to storage_pool table. This will be populated while creating zone wide primary storage pool.
Currently VMware and KVM are supported.

The API changes:

1. When adding a storage into cloudstack, admin needs to specify which scope he wants to add it into, either cluster or zone. Thus CreateStoragePoolCmd will have an extra parameter, called, scope.

If the scope is null, which means the scope is cluster, for api back-compatibility. 

If the scope is ZONE, hypervisor parameter would be mandatory parameter for createStoragePool. If scope is not ZONE then hypervisor parameter is optional, hence api backward compatibility is taken care.

2. A new parameter, scope, will be added into ListStoragePoolsCmd, in order to list available storages in cloudstack. 

If the scope is null, which means list storage pool whose scope attribute is cluster, for api back-compatibility.

The flow of how to register a zone-wide primary storage:

1. CreateStoragePoolCmd with scope=zone & hypervisor=VMware or hypervisor=KVM

2. add storage pool into database

3. if there are kvm hosts in up state, send down a modifystoragepoolcmd to each kvm hosts in the zone

4. Then finished

5. Whenever a new kvm hosts added into the zone, also send down modifystoragepoolcmd to the host.

The change in storage allocator:

If there are zone-wide storages available in the zone, storage allocator will use the following algorithm:

1. Only use zone-wide storage, if it's for kvm hypervisor

2. If there are both cluster/host wide storages available for a kvm host, choose cluster or host wide storage at first.

3. If there is only zone-wide storages available for a kvm host, then choose one of storage, based on first-fit algorithm

In zone wide storage pool allocator, while finding suitable pools for volume, hypervisor type of pool would be compared against hypervisor type from disk profile.

  • No labels