• Created by Unknown User (weizhou), last updated on Aug 22, 2024 5 minute read


This is mostly written by Abhisar

Project Introduction

Add support for shared storage file system or File Share or Storage Share as a first class feature in Cloudstack. The File Share can be attached to a guest network andcan be used by any VM on the network in a shared manner. The File Share is designed to be resizable and highly available.

Benefits

this feature can later be used as integration end-points with the CSI driver, go-sdk, Terraform, Ansible and others.

Future extensions

If this feature is done as plugin-framework model (or expanded later on), other vendors can implement their solutions in future to support Ceph/CephFS, Trunas, Netapp filers etc that can provide native FS to user VMs.

Terminology

FileShare : Deployment of a shareable file system which can be mounted by VMs on the given network.

StorageFsVM : Specialised system VM acting as the server for the file system.

Functional Specification

Functionality wise, users can

  • Create a file share with the specified service offering, disk offering and network.

  • Admins can create the file share with the specified owner (account or project)

  • Users can also specify which filesystem format (EXT4, XFS) to use.

  • List and update details about the file share.

  • Change disk offering (Size can only increase in this version)

  • Change service offering

  • Restart the file share with or without cleanup. Restart with cleanup will reinstall the fileshare VM.

  • See File Share utilisation and other disk stats.

  • Destroy the file share with an expunge delay.

Scope and Limitations

These limitations are specific to this release and can be removed in future.

  • Only NFS protocol support.

  • The stats that are displayed are taken from the hypervisor and the data volume. No direct communication with the VM to get metrics from inside the VM.

  • File share cannot be moved between accounts/domain/project

  • File share doesn’t use storage tags. Data volume can’t be deployed on a specified primary storage.

  • Networks with Security groups are not supported.

StorageFsVm File Share provider

There can be multiple File Share providers in the future, but for the current release, there is only one provider which is basically a user VM with a new user vm type STORAGSFSVM. A data volume is attached to the VM on which the filesystem is installed. Like any user VM, it is only attached to the guest network. Userdata is used to push some udev rules to the VM.
These udev rules take care of following operations :

  • Format and export the FS, the first time a data volume is added to the VM

  • Resize the FS whenever the volume is resized

There is no communication between the management server and the VM which makes this implementation hypervisor agnostic. We can think of moving to a agent based model in the future if more features which can not be implemented using udev.

The VM uses user service offerings but with some checks put on it (min cpu, min ram size and ha-enabled).

The VM is installed with the SystemVM template. Users also have the option to do ‘Restart with cleanup’ which will reinstall the fileshare VM to serve the NFS share. This action can be used for troubleshooting and for upgrading the Systemvm template.

The File Share VM can be seen in the Instance Tab as well. It’s name is prefixed by the File Share name. Actions that might interfere with File Share operations are blocked or not shown. Basic operaions like Start, Stop and Reboot are allowed for troubleshooting. Users can access the VM using the ‘View Console’ button for troubleshooting although it is not recommended during normal operations.

New APIs

APIs on FileShare

API

Parameters

Description

createFileShare

name
description
account
domain
projectid
diskoffering
serviceoffering
fsformat
fileshareprovider
networkid

Create a new File Share with the given size, disk offering, service offering, file system format and network.

listFileShare

id
name
network
diskoffering
serviceoffering

list File Shares by ids otr list all File Shares on the given networkids or list all File Shares by account

destroyFileShare

id
expunge

destroys a File share by the given id. Set expunge equal to true for force expunge.

recoverFileShare

id

recover a destroyed file share

deleteFileShare

id

expunge the destroyed file share

startFileShare

id

start the StoragefsVM

stopFileShare

id

stop the StoragefsVM

restartFileShare

id
cleanup

Reboots the File Share VM. If cleanup is true, the fileshare VM is reinstalled.

updateFileShare

id
name

Update name and description of the file share

changeDiskOffering

id
diskOfferingId
size
miniops
maxiops

Change the disk offering of the file share. Extra parameters like size, miniops and maxiops can be provided if the disk offering takes custom parameters

changeServiceOffering

id
serviceOfferingId

Change the service offering for the file share. This will upgrade the fileshare VM with the new service offering and mount the FIleshare on that VM.

APIs on providers

API

Parameters

Description

listFileShareProviders


List all the available File Share Providers.

Enums

FSFormat : EXT4, XFS.

New Global Settings

Setting

Description

fileshare.feature.enabled

Indicates whether the File Share feature is enabled or not. Management server restart needed on change
Default : Enabled

fileshare.cleanup.delay

Determines how long (in seconds) to wait before actually expunging destroyed file shares.
Default : 86400

fileshare.cleanup.interval

The interval (in seconds) to wait before running the fileshare cleanup thread.
Default : 14400

storagefsvm.min.cpu.count

Minimum cpu count allowed for the compute offering to be used to create storagefsvm for file shares.
Default : 2

storagefsvm.min.ram.size

minimum ram size allowed for the compute offering to be used to create storagefsvm for file shares
Default : 1024

File Share States and FSM

States

State

Description

Allocated

The file share is allocated in DB but hasn’t been created or started yet.

Ready

The file share is ready to use.

Stopping

The file share is being stoppped.

Stopped

The file share is in stopped state it cannot be used but the data is still there.

Starting

The file share is being started.

Detached

The file share data is not attached to any VM.

Destroyed

The file share is destroyed.

Expunging

The file share is being expunged.

Expunged

The file share has been expunged

Error

The file share is in error state.

Events

  • StopRequested

  • StartRequested

  • Detach

  • DestroyRequested

  • OperationSucceeded

  • OperationFailed

  • ExpungeOperation

  • RecoveryRequested

FSM

Old State

Event

New State

Allocated

OperationFailed

Error

Allocated

OperationSucceeded

Stopped

Error

DestroyRequested

Destroyed

Stopped

StartRequested

Starting

Starting

OperationFailed

Stopped

Starting

OperationSucceeded

Ready

Detached

OperationSucceeded

Stopped

Detached

OperationFailed

Detached

Ready

StopRequested

Stopping

Stopping

OperationSucceeded

Stopped

Stopping

OperationFailed

Ready

Stopped

Detach

Detached

Stopped

DestroyRequested

Destroyed

Destroyed

RecoveryRequested

Stopped

Destroyed

ExpungeOperation

Expunging

Expunging

ExpungeOperation

Expunging

Expunging

OperationSucceeded

Expunged

Database Changes

New Tables

  1. storage_file_share

Column

Type

Comment

id

bigint

id of the file share

uuid

varchar(255)

uuid of the file share

name

varchar(255)

name of the file share

description

varchar(1024)

description of the file share

domain_id

bigint

id of the owner domain of the file share

account_id

bigint

id of the owner account of the file share

data_center_id

bigint

id of the zone

state

varchar(10)

current state of the file share

fs_provider_name

varchar(255)

name of the file share provider

Protocol

varchar(10)

Protocol supported by the file share

volume_id

bigint

id of the data volume on which filesystem is installed

vm_id

bigint

id of the vm where the file share is deployed

fs_type

varchar(10)

file system which will be installed on the file share

service_offering_id

bigint

service offering to use for the storagevm

updated

datetime

last updated. Used by fsm.

created

datetime

date created

removed

datetime

date removed

Code Organisation

API layer

All user and admin APIs are implemented in these directory : api/src/main/java/org/apache/cloudstack/api/command/user/storage/fileshare/

api/src/main/java/org/apache/cloudstack/api/command/admin/storage/fileshare/

Service layer

FileShareManager interface in : api/src/main/java/org/apache/cloudstack/storage/fileshare/

FileShareManagerImpl in : server/src/main/java/org/apache/cloudstack/storage/fileshare/
Service class for all File Share related APIs. Also provides framework definition for registering and retrieving all plugins

Plugins

File Share providers are added to plugins/storage/fileshare

For the current release we have only one plugin storagefsvm is implemented in the directory : plugins/storage/fileshare/storagefsvm

Dao and VO

Dao and VO classes defined in schema/src/main/java/org/apache/cloudstack/storage/fileshare

  • No labels