Page History

...

1.  Introduce `CLVM` and `CLVM_NG` as first-class `StoragePoolType` entries in CloudStack. CLVM supports RAW volumes on LVM and CLVM_NG support QCOW2.
2. Design a lock-tracking layer that records and reconciles which KVM host holds the exclusive LVM activation for each volume.
3. Implement a lock-transfer mechanism so CloudStack can safely move the exclusive lock between hosts without copying data.
4. Introduce a three-tier migration strategy that avoids unnecessary data copying:
      - Lock Transfer - same pool, wrong host: transfer lock only.
      - Lightweight Migration - different CloudStack pool records sharing the same LVM Volume Group: transfer lock only.
      - Full Migration - different Volume Groups: full data copy (existing behavior).
5. Provide a secure-erase option for LVs at deletion time to prevent data leakage.
6. Incremental snapshots for CLVM_NG leveraging bitmap

LVM Activation Modes

Architecture Overview:

┌─────────────────────────────────────────────────────────────┐
│              CloudStack Management Server                    │
└──────────────────────┬──────────────────────────────────────┘
                       │
    ┌──────────────────┼──────────────────┐
    │                  │                  │
┌───▼─────────┐  ┌────▼──────┐  ┌───────▼────────┐
│  VolumeApi  │  │  Volume   │  │     Default    │
│ ServiceImpl │  │Orchestrator│  │EndPointSelector│
│             │  │            │  │                │
│ • attach    │  │ • VM start │  │ • Route ops    │
│ • detach    │  │ • migrate  │  │   to lock host │
│ • migrate   │  │ • prepare  │  │ • Query lock   │
└─────────────┘  └────────────┘  └────────────────┘
       │                │                │
       └────────────────┼────────────────┘
                        │
              ┌─────────▼─────────┐
              │ VolumeServiceImpl │
              │                   │
              │ • performLock     │
              │   Migration       │
              │ • findLockHost    │
              │ • isLightweight   │
              │   Needed          │
              └─────────┬─────────┘
                        │
              ┌─────────▼─────────┐
              │  ClvmLockManager  │
              │                   │
              │ • transferVolume  │
              │   Lock            │
              │ • queryCurrent    │
              │   LockHolder      │
              │ • setClvmLock     │
              │   HostId          │
              └─────────┬─────────┘
                        │ (AgentManager)
                        │
        ┌───────────────┴───────────────┐
        │                               │
┌───────▼────────┐             ┌───────▼────────┐
│  KVM Agent 1   │             │  KVM Agent N   │
│  (Host A)      │             │  (Host B)      │
│                │             │                │
│  LibvirtClvm   │             │  LibvirtClvm   │
│  LockTransfer  │             │  LockTransfer  │
│  Wrapper       │             │  Wrapper       │
│                │             │                │
│  lvchange -an  │             │  lvchange -aey │
│  lvchange -aey │             │  lvs -o ...    │
│  lvs           │             │                │
└────────────────┘             └────────────────┘
        │                               │
        └───────────────┬───────────────┘
                        │
            ┌───────────▼────────────┐
            │  Shared Storage (SAN)  │
            │  LVM Volume Group      │
            │  /dev/vg-cluster01/... │
            └────────────────────────┘

...

• ClvmLockTransferCommand / ClvmLockTransferAnswer
  1. Operations:
     

     Code Block
     public enum Operation { DEACTIVATE("-an", "deactivate"), ACTIVATE_EXCLUSIVE("-aey", "activate exclusively"), ACTIVATE_SHARED("-asy", "activate in shared mode"), QUERY_LOCK_STATE("query", "query lock state"); }

     
     

  2. Command Properties

     Property	Value	Description
     executeInSequence()	true	Prevents concurrent lock operations on same LV
     setWait(30)	30 secs	Agent timeout per operation
     Fields	lvPath, operation, volumeUuid	command parameters

     
     

  3. Answer Fields
     

     Field	Type	Description
     currentLockHostname	String	Hostname holding the lock from lv_host field
     isActive	boolean	Whether LV is active anywhere
     isExclusive	boolean	Weather the active lock is exclusive
     lvAttributes	String	Raw lv_attr string for debugging

     
     
     
• ClvmLockManager
  

  getClvmLockHostId(volumeId, volumeUuid, volumePath, pool, queryActual)

  Purpose:Returns the host ID holding the exclusive lock
  
  

  Code Block
  if queryActual == true: return queryCurrentLockHolder(...) // Query LVM directly else: return database value from volume_details.clvmLockHostId

  
  
  Parameters:

• queryActual=true: Bypass database, query LVM state (expensive but accurate)

• queryActual=false: Fast database lookup (may be stale)

...

VirtualMachineManagerImpl.migrate()

1. Pre-migration checks and preparation:
├─ volumeMgr.prepareForMigration(profile, dest)
├─ Generate MigrateCommand with VM and disk information
├─ PreMigrationCommand already executed (Phase 0: Source → SHARED)
└─ PrepareForMigrationCommand already executed (Phase 1: Dest → SHARED)

2. Initiate libvirt live migration:
├─ Command: domain.migrate(destConn, xmlDesc, migrateFlags)
├─ Libvirt uses shared storage
├─ Transfers memory state over network
├─ QEMU on destination opens block devices
└─ BOTH hosts have volumes in SHARED mode (dual activation)

3. After successful migration:
├─ Deactivate CLVM volumes on source host:
│ LibvirtComputingResource.modifyClvmVolumesStateForMigration(
│ disks, resource, vmSpec, ClvmVolumeState.DEACTIVATE)
│ → executes: lvchange -an /dev/<vg>/<lv> on source
│ → Destination volumes remain ACTIVE in SHARED mode
│
└─ Update CloudStack database tracking:
updateClvmLockHostForVmVolumes(vm.getId(), destHost.getId())
→ sets CLVM_LOCK_HOST_ID = destHost for all volumes

4. On migration failure:
├─ Revert CLVM volumes to EXCLUSIVE mode on source:
│ LibvirtComputingResource.modifyClvmVolumesStateForMigration(
│ disks, resource, vmSpec, ClvmVolumeState.EXCLUSIVE)
│ → executes: lvchange -aey /dev/<vg>/<lv> on source
│
└─ Destination volumes are deactivated/cleaned up via rollback

...

cloudmonkey create storagepool \
  zoneid=<zone-id> \
  podid=<pod-id> \
  clusterid=<cluster-id> \
  name="clvm-pool-01" \
  url="clvm:///<vg-name>" \
  scope=cluster



cloudmonkey create storagepool \
  zoneid=<zone-id> \
  podid=<pod-id> \
  clusterid=<cluster-id> \
  name="clvm-ng-pool-01" \
  url="clvm_ng:///<vg-name>" \
  scope=cluster

Global Setting

When enabled, KVM agents execute dd if=/dev/zero of=<lv> before lvremove. This prevents the next VM allocated to that storage space from reading previous tenant data.

...