Page History

Table of Contents

Introduction

An Object Storage service can store a large amount of unstructured data of any content type. Object Stores are widely used for Big Data analytics, Backup/Archival, Web Hosting and media storage and delivery. Amazon S3, Google Cloud Storage, Azure Blob Storage and DigitalOcean Spaces are some of the popular Object Storage service providers. 

Building on the OpenSource object store providers, CloudStack can support Object Storage as a first-class entity. A general purpose object storage framework should be added, with support for various Object Storage providers as plugins. CloudStack will manage the lifecycle of the object storage pools and buckets. Objects will not be managed by CloudStack. The underlying object-storage provider is responsible for flexible storage management and data growth, multi-site replication, security, and backup.

Purpose

The purpose if this document is to provide functional specification for Object Storage feature in CloudStack.

References

• relevant linkshttps://min.io/

Document History

Glossary

Object Store: Object storage (also known as object-based storage) is a data storage that manages data as objects

...

• Object storage as a first-class end user feature in CloudStack

• Pluggable Object storage provider framework. MinIO will added as a supported provider in the initial implementation 

• Ability to create buckets and manage their lifecycleAble to create/manage buckets

Use cases

• CloudStack admin should be able to add supported Object Storage providers (min.io based infra, ceph object storage etc.).
• CloudStack users should be able to create buckets and manage their lifecycle.
• Users/projects get billed by aggregate data storage on per bucket basis

Architecture and Design description

ObjectStoreDriver interface will be defined. All supported object store providers will need to implement this interface. 

API and Schema Changes

New APIs

Object Storage Pool APIs

• addObjectStoragePool

  • name
  • description
  • URL
  • Provider
  • Details - Map

• updateObjectStoragePool

  • id
  • name
  • description

• removeObjectStoragePool

  • id

• listObjectStoragePools

  • id

Bucket APIs

• createBucket

  • name
  • description
  • object_storage_id
  • encryption
  • versioning
  • object_lock
  • policy

• deleteBucket

  • id

• listBuckets

  • id
  • provider
  • object_storage_id
• updateBucket
  • id
  • name
  • description
  • encryption
  • versioning
  • object_lock
  • policy

Schema Changes

New Tables:

• object_store
  • id
  • uuid
  • name
  • description
  • provider
  • url
  • total_size
  • used_size
  • created
  • removed
• object_store_details
  • id
  • store_id
  • name
  • value
• bucket
  • id
  • uuid
  • name
  • description
  • store_id
  • size
  • url
  • created
  • removed

Service Layer Changes

Describe business/service layer changes, changes in frameworks, plugins ...<ToDo>

UI Changes

• UI for managing object storage pools
• UI for bucket operationoperations
• S3 compatible UI client for managing objects

Usage Impact

• Buckets usage will be added as a new Usage resource type

  • Size of the bucket will be included as key metric for billing purposes

Appendix

Appendix A:

Appendix B: