THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Bug Reference
Branch
master, 4.2.0
Introduction
Dedicating pod, cluster or host to a specific domain/account means that the domain/account will have sole access to the dedicated pod, cluster or hosts such that scalability, security and manageability within a domain/account can be improved. The resources which belong to that tenant, will be placed into that dedicated pod, cluster or host.
Current Scenario
- Currently in CloudStack, zones can be reserved for specific domains. Only users in that domain or its subdomain may create guests in that zone.
- Dedicated Hosts and HA Hosts: if one of the dedicated hosts fail then the VMs are HAed onto a specific host(s) that is dedicated for purposes of HA.
- Domains/Accounts cannot have private pod, private cluster or private host
Purpose
Dedicating a zone might be very expensive offering for several end-users whereas dedicating a pod/cluster/host may be more economical. This feature will allow root admin to dedicate resources to a specific domain/account that needs private infrastructure for additional security or performance guarantees.
This document describes the specifications and design of this feature.
References
Preliminaries
1. Explicitly Dedicated Resources: Resources dedicated to an account during configuration time
2. Implicitly Dedicated Resources: Resources which are in common pool that any account can pick during runtime.
3. Shared Resources: All the non-dedicated resources.
4. SO dedication flag: a new Parameter in Service Offering. if ON, Use Implicitly Dedicated Resources
5. DVM dedication flag: a new Parameter in deployVirtualMachine API, if ON, VM is deployed on explicitly dedicated resources
This new parameter has been removed as per discussion.
An "explicit dedication" type will be added in Affinity type which will indicate deployment on explicitly dedicated resources.
Feature Specifications
Requirements
- Root Admins must be able to explicitly dedicate a zone, pod or cluster or host to a specific domain or account
- When deploying a VM, end user can optionally specify it to be placed on a dedicated resource
- Explicit dedication can be for a domain or account and can be a zone/pod/cluster/host; for example, if a cluster is explicitly dedicated to a domain, any account in that domain can share the hosts (as long as they DeployVM with dedication flag on), but sharing is limited to accounts within that domain only. As another example, if a cluster is explicitly dedicated to an account, only that account’s VM can use those hosts in the dedicated cluster (as long as they DeployVM with dedication flag on).
- Alternatively, a host or cluster or pod or zone can be "implicitly" dedicated - this flag's description will be clarified below
- Implicit dedication can be for a zone/pod/cluster/host, but not associated with any domain or account; however, during VM deployment, once chosen, a host will not be shared across multiple accounts – as an example, here is a reason: for deployment of certain types of applications, such as desktops, due to licensing reasons, no host can be shared between different accounts. However, pre-allocating specific (number of) hosts to each account is not desired as this will create forecasting issues/sub-optimal utilization
- A service offering can optionally have an "implicitly" dedicated flag on - this flag's description will be clarified below in the table
- Admin must be able to live migrate of such a VM to a resource not owned by the account, but an alert must be generated
- Fail the request if DeployVM dedication Flag is specified and SO has implicit dedication ON
- When you search implicitly dedicated “area” for a host (second table below), if there is already a host for this account, pick it up (instead of finding a new one every time)
- When you delete the last VM from a host in the implicit area, remove the association of the account with the host (i.e. make it implicit-free host as before)
- Table 1:Deploying VM with DeployVM dedication flag
DeployVM dedication Flag specified
DeployVM dedication Flag Not specified
Explicitly Dedicated Resources
Pick from Explicitly dedicated else fail
Pick from shared resources else fail
NO Explicitly Dedicated Resources
Fail the request
Pick from shared resources else fail
- Table 2: Deploying VM with SO dedication flag
SO dedication Flag = ON
SO dedication Flag = OFF
YES - Implicitly Dedicated Resources
Pick from implicit dedicated resources else FAIL
Pick from shared resources else fail
NO - Implicitly Dedicated Resources
Fail the request
Pick from shared resources else fail
Logs
- Ensure proper logs are maintained into vmops.log and api.log
Test Guidelines
- Dedicated resources can only be used if service offering dedication flag is ON or DeployVM dedication flag is ON but not both.
User Permissions
- Only Root Admin can dedicate zone, pod, cluster or host to specific domain or an account.
- Only Root admin can implicitly dedicate a zone, pod, cluster or a host.
- If a user does not belong to a domain which has dedicated resources, he cannot access the pod, cluster or host dedicated to that domain/account.
- Users belonging to domain/account having dedicated resources, can access them but should not be allowed to modify.
- At this time, there is no requirement for the domain/sub-domain admins to manage the resource - the root admin will remain the owner of the resources
- Only Root Admin can add a service offering with "isdedicated" option enabled.
- User/admin can enable the DeplovVM dedication flag.
Use Cases:
VM Deployment
The following figure graphically illustrates the allocation of hosts for VM deployment.
Host 1 is explicitly dedicated to domain D1, Host 2 is implicitly dedicated, and Host 3 is a shared host.
SO is the Service Offering Dedication Flag and DVM is the DeployVM Dedication flag.
Use Case 1: User U1 of Account A1, deploys a virtual machine VM1 with SO=ON,
Host 2 is chosen from the pool of implicitly dedicated resources (In this case, only Host 2 is implicitly dedicated), Implicitly Dedicated "Host 2" is now dedicated to account A1
Use Case 2: User U1 of Account A1, deploys a virtual machine VM2 with DVM=ON,
Host 1 is chosen form the pool of explicitly dedicated resources (In this case, only Host 1 is explicitly dedicated)
Use Case 3: User U1 of Account A1, deploys a virtual machine VM3 without any flag ON,
Host 3 is chosen from the shared pool, (In this case, only Host 3 is a shared Host)
Use Case 4: User U3 of Account A2, deploys a virtual machine VM1 with SO=ON,
Request Failed, No Host available
Use Case 5: User U4 of account A2, deploys a virtual machine VM2 with DVM=ON,
Host 1 is chosen form the pool of explicitly dedicated resources
Use Case 6: User U5 of Account A3, deploys a virtual machine VM1 with SO=ON,
Request Failed, No Host available
Use Case 7: User U5 of Account A3, deploys a virtual machine VM2 with DVM=ON,
Request Failed, No Host available.
Use Case 8: User U5 of Account, deploys a virtual machine VM3 without any flag,
Host 3 is selected.
Use Case 9: Once All the VMs in Host 2 is deleted, Host will added to the implicit dedicated pool.
Use Case 10: Host 1 will remain dedicated to Domain D1 unless root admin changes dedication of this host.
Dedicating resources to Domain/Account: Explicit Dedication
Domain level accessibility:
Let D1 domain has SD1, SD2, SD3 sub-domains. A1 is the admin account, U2 is normal user account.
Here z1, z2, ... are zones , p1,p1, ... are pods, c1, c11, c2 ,.... are clusters, H1, H2, ... are Hosts
z1 - (p1-c1,c11,c111),(p2-c2,c22),(p3-c3, c33)
z2 - (p4-c4)
- Root Admin should be able to dedicate a pod, cluster or host to any domain or sub-domain.
- Once a Zone is dedicated to a domain, its pods and clusters will be dedicated to that domain eg: pod: p1 is dedicated to domain D1, clusters:c1, c11, c111 will be automatically dedicated to D1 by default.
- Users in Sub-Domains SD1, SD2, SD3 should be able to deploy vm in parent domain's clusters c1, c11, c111 or pod p1.
- After dedicating pod p1 to domain D1, if further cluster c11 (in pod p1) is dedicated to SD3, then D1 or SD1 or SD2 should not be able to access c11. (Can SD3 use SD2, SD1 or D1's resources, verify)
- If another pod p2 is dedicated to SD1, then SD11, SD12 or D2 should not be able to access pod p2.
- Before dedicating a pod to a domain , check whether its zone is dedicated or not.
- Child Domain can access pod/cluster/host dedicated to parent domain, vice-versa is not (TBD).
Account level accessibility:
- Once a pod/cluster/host is dedicated to an account, only users in that account can access it.
- No other user from different accounts in the same domain or different domain can access the resources.
Deletion of Account/domain
- Deleting an account will delete all the VMs, snapshots, templates, etc. of that account, and also removes the dedication of host, cluster or pod to that account.
- Deleting a domain, will remove the dedication from the hosts, clusters or pods (if dedicated).
Host Tags with SO Dedication Flag and DeployVm dedication Flag
- If Host Tag is provided and SO Dedication Flag is ON and DeployVm dedication Flag OFF
- Host which is tagged as well as implicitly dedicated, will be allocated. If not found, fail the request.
- If host tag is provided, SO Dedication Flag OFF and DeployVm dedication Flag OFF
- Host which is tagged but not implicitly or explicitly dedicated, will be allocated. If not found, fail the request.
- If host tag is provided, SO Dedication Flag OFF and DeployVm dedication Flag ON
- Host which is tagged but not implicitly or explicitly dedicated, will be allocated. If not found, fail the request.
Migration of VMs
- If VM to be migrated is non-dedicated
- if destination host is implicitly or explicitly dedicated, migrate it to that host, but an alert is generated.
- if destination host is non-dedicated, migrate it to the destination host.
- If VM to be migrated is dedicated
- if destination host is implicitly or explicitly dedicated to the same account, migrate it to that host.
- if destination host is implicitly or explicitly dedicated to different account, migrate it to that host, but an alert is generated
- if destination host is non-dedicated, migrate it to that host but an alert is generated.
Architecture and Design description
This feature will come as a separate plugin which will use the PluggableService to add the DedicatedResources feature APIs into CloudStack, the DedicatedResourceManager will be responsible for thread scheduling and the DedicatedResourcePlanner which will implement a deployment planner interface.
Explicit Dedication:
- New Admin APIs to dedicate Zones/Pods/Clusters/Hosts to a domain or account (these APIs will come in a separate plugin)
- list affinity types: Add a new type: explicit dedication
- User should be able to create affinity groups based on the explicit dedication type
- User can associate above affinity group to VM during deployment.
- A new ExplicitDedicationProcessor plugin that implements AffinityGroupProcessor adapter, will set deployment plan scope to the correct resource level (For AffinityGroupProcessor adapter see: https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups).
- The Deployment Planning Manager will do the following:
- Processors: Call the ExplicitDedicationProcessor to process the dedication request based on the explicitly dedicated resources for the user.
- Planners: Call Deployment Planner to drill down to the right set of clusters to look for placement based on explicitly dedicated clusters/hosts inside them.
- Allocators: Call Allocators to match the requirement to explicit dedication.
New Addition: ExplicitDedicationProcessor
This will implement the process to set scope of deployment plan to the correct resource level for planners to look into. This processor will implement AffinityGroupProcessor adapter.
For Example:
Zone Z has Pods P1, P2.
Pod P1 has Clusters C1, C2 and P2 has Cluster C3
Clusters
C1 -> h1 (dedicated A), h2 (dedicated B)
C2 -> h3 (dedicated A)
C3 -> h4, h5(dedicated B)
Case 1: Deploy VM for account A with explicit dedication, processors should set the scope of plan to Pod P1 since both dedicated hosts are in Pod P1 even if in different clusters. Also set the Pod P2 in avoid set
Case 2: Deploy VM for account B with explicit dedication, processors should set the scope of plan to Zone Z since the dedicated hosts are in Pod P1 and P2 , with parent being the zone itself. Also set the Cluster C2 in avoid set.
Modify Planners and Allocators
Existing Planners and Allocators will be modified to make sure that dedicated resources are not picked, when dedication is not required.
Implicit Dedication:
Still in discussion...
Change in the Existing Private Zone functionality:
Zones can be dedicated to domains while creation. Users in that domain can deployVm only in that dedicated zone.
Change:
- Zones dedicated to a domain at the creation level will be considered as explicit dedication of zones. The functionality will be redirected to dedication affinity group when Vm (with explicit dedication) is deployed.
- All the Zones will be listed, and if explicit type affinity group is provided, deployment will only happen if any of the resource in that zone will be dedicated to that user's account or domain.
API Changes
Following new APIs will be added:
- Dedicating Resources (Explicit or Implicit)
- dedicateZone
- dedicatePod
- dedicateCluster
- dedicateHost
These are new admin APIs to dedicate a zone/pod/cluster/host for an account/domain.
Parameters include:
a) ZoneId/PodId/ClusterId/HostId
b) DomainId
c) AccountId
d) ImplicitDedication
Note: Either "DomainId or AccountId" OR "ImplicitDedication Flag" can be provided as a parameter but not both. If neither of them specified, fail the request.
- Release dedication of resources
- releaseDedicatedZone
- releaseDedicatedPod
- releaseDedicatedCluster
- releaseDedicatedHost
These are new admin APIs to remove the dedication of zone/pod/cluster/host
Parameters include:
a) ZoneId/PodId/ClusterId/HostId
- Listing Dedicated resources per account/domain
- listDedicatedZones
- listDedicatedPods
- listDedicatedClusters
- listDedicatedHosts
These are new admin APIs to list dedicated pods/clusters/hosts for an account/domain.
Parameters include:
a) zoneId/PodId/ClusterId/HostId
b) DomainId
c) AccountId
d) ImplicitDedication flag
Existing API modification:
- createServiceOffering: Request Parameter Addition:
Parameter Name |
Description |
Required |
|---|---|---|
isdedicated |
if true, Instance will be deployed on the host dedicated to the account |
false |
DB Changes:
- New Table : dedicated_resources (id, uuid, zone_id, pod_id, cluster_id, host_id, domain_id, account_id)
|| Field || Type || Null || Key || Default || Extra ||id
bigint(20) unsigned
NO
PRI
NULL
auto_increment
uuid
varchar(40)
YES
UNI
NULL
data_center_id
bigint(20) unsigned
NO
MUL
NULL
pod_id
bigint(20) unsigned
NO
MUL
NULL
cluster_id
bigint(20) unsigned
NO
MUL
NULL
host_id
bigint(20) unsigned
NO
MUL
NULL
domain_id
bigint(20) unsigned
YES
MUL
NULL
account_id
bigint(20) unsigned
YES
MUL
NULL
implicit_dedication
tinyint(1) unsigned
NO
MUL
0
- service_offering table: Introduce a column “isdedicated” in service_offering table. Default value should be 0.
UI flow
- Add a check option in "Add Compute Offering" for is_dedicated flag.
Overview
Content Tools
Apps