Zone-wide primary storage target

Background: When a data disk is needed by a VM that resides in a different cluster (i.e. different primary storage) than the current cluster/primary storage, it involves a costly/time consuming data copy operation - which involved copying data from one primary storage to another primary storage via the secondary storage. A primary storage is not shared across multiple clusters. As an example, AWS EBS provides the following benefits: EBS volumes are placed in a specific Availability Zone, and can then be attached to any instance in that same Availability Zone. Even though a volume can only be attached to one instance at a time, if an instance fails or is detached from an EBS volume, the volume can be attached to any other instance in that Availability Zone.

Requirement: CloudStack to support zone-wide (primary) block storage

- Multiple clusters can share the same primary storage (as long as the underlying hypervisor platform supports it)

- Only one VM can mount/attach a disk at any time (i.e. multiple VMs cannot attach the same disk concurrently)

- snapshot/restore features are similar to current semantics i.e. no degradation in features

- we should allow multiple zone-wide primary storages

-A cluster can have a cluster-specific and zone wide primary storage concurrently

- must support multiple hypervisors (XS/VMware for example) per zone

- upon migration, existing customers with more than 1 cluster per zone must not have any impact – any upgrade process needs to be clearly documented

- During the upgrade, no need to allow promotion of one primary storage to become the zone-wide primary storage;

- Adding zone-wide primary to existing zone will need to be possible

-          In addition to each current primary storage

-          In lieu of current primary – in this case, storage must be migrated manually, prior to deleting the existing primary storage

- Disk offering setting will now have choices for the admin on where the disk will be created – local, cluster-wide or zone-widestorage

 - How to migrate each primary storage data to “new” zone-wide – how can this be less painful - need to be clarified

Design

In the new storage framework, each storage can have its own life cycle, the storage will be not tied to a cluster or a host. 

The DB changes:

1. new column, called scope, will be added into storage_pool table. The scope has following attributes:

    HOST, CLUSTER, ZONE;

Host means the storage is attached to a particular hypervisor host, a.k.a, it's the local storage.

Cluster, means the storage is attached to a cluster, it's the primary storage we have today.

Zone, means the storage is attached to a zone, 

The API changes:

1. When adding a storage into cloudstack, admin needs to specify which scope he wants to add it into, either cluster or zone. Thus CreateStoragePoolCmd will have an extra parameter, called, scope.

If the scope is null, which means the scope is cluster, for api back-compatibility. 

2. A new parameter, scope, will be added into ListStoragePoolsCmd, in order to list available storages in cloudstack. 

If the scope is null, which means list storage pool whose scope attribute is cluster, for api back-compatibility.

The flow of how to register a zone-wide primary storage:

1. CreateStoragePoolCmd with scope=zone

2. add storage pool into dabase

3. if there are kvm hosts in up state, send down a modifystoragepoolcmd to each kvm hosts in the zone

4. Then finished

5. Whenever a new kvm hosts added into the zone, also send down modifystoragepoolcmd to the host.

The change in storage allocator:

If there are zone-wide storages available in the zone, storage allocator will use the following algorithm:

1. Only use zone-wide storage, if it's for kvm hypervisor

2. If there are both cluster/host wide storages available for a kvm host, choose cluster or host wide storage at first.

3. If there is only zone-wide storages available for a kvm host, then choose one of storage, based on first-fit algorithm