Storage Backup Object Store Plugin Framework

Problem Statement

Currently CloudStack does not offer a flexible pluggable framework for users to easily integrate and configure any 3rd-party object stores for such backup services as registering templates, taking snapshots, etc. Along with Edison's recent refactored storage subsystem 2.0, we are proposing to develop a storage backup object store plugin framework to allow CloudStack to systematically manage and configure various types of backup data stores from different vendors.

Storage DataStore Plugin Framework

In this new refactored backup datastore plugin framework, we have clearly defined those pluggable service interfaces, such as PrimaryDataStore, ImageDataStore, DataMotionStrategy, AutoScaleStrategy, etc, so that different storage providers can develop their vendor-specific plugins based on the well-defined contracts that can be seemlessly managed by CloudStack orchestration.

Benefits of Backup DataStore Plugin Framework

With this new plugin framework, we should be able to achieve following functionalities:

Support different object store provides in a uniform and extensible fashion

Although current CloudStack has some existing business logic to handle S3, Swift, NFS secondary storages, the code to interact with these vendor-specific providers is closely mingled together with CloudStack orchestration code, specifically, TemplateManagerImpl, SnapshotManagerImpl, etc, using an ugly if-else control flow. With such a tight coupling, it becomes very hard to extend the framework to support new object store providers. By extracting out ImageDataStore pluggble service interface, we can wrap these vendor-specific logic inside each provider plugin, then CloudStack orchestration code can be greatly simplified and provide a uniform and extensible way to manage different object store vendors.

Enable region wide object backup.

Currently CloudStack only allows registering a template to a specific zone by using NFS secondary storage. In order to enable this template to another zone, user has to invoke copyTemplate to copy the template from one zone to another. With this new plugin framework, customers can plugin their own object store providers, like S3/Swift, etc and choose them as the backup store. CloudStack can provide an option in *registerTemplate* api to allow them to create either "perZone" or "regionWide" template.

instead of currently copying templates from one zone to another.

Make SSVM optional and configurable

Support pluggable Data Motion Strategies to handle various protocol combinations.

Provide CloudStack framework support to launch NFS transfer VMs dynamically for data transfer.

Support pluggable Auto Scale strategies to load balance data streams.