Page History

...

• We are going to re-use existing Async Job management logic with a couple of new additions for createSnapshot scenario. Other async jobs won't get affected.
• createSnapshot API now going to be synchronized on Host object. The host object would be determined even before the command is scheduled for execution - using cloud.host table information (host_id field for snapshots created for vms in Running state, and last_host_id for vms in Stopped state)

...

Configuration parameters changes.

...

• 1 : 2 new global config parameters are added: concurrent.snapshots.threshold.perhost and createsnapshot.job.cancel.threshold.minutes (controls job expiration time).

...

DB changes

...

sync_queue table

•  new fields queue_size and : new field queue_size is added to cloud.sync_queue table (_limit are added  (first determines how many jobs are being processed at the moment), second - the max number of jobs that can be processed at the same time). The default value for the field queue_size is 0. Can't be greater than 1 for any job Field queue_size_limit is 1 for  all jobs other than createSnapshot. For createSnapshot the value can't get higher than the parameter its determined by concurrent.snapshots.threshold.perhost allows. New filed job_timeout is added to async_job table - determines the timeout for the async job. If NULL, the timeout is defaulted to the global config value (60 mins) that is respected by any async job. This parameter would be set only for createSnapshot jobs at the moment - to the value defined in createsnapshot.job.cancel.threshold.minutes.Code flow: .
• removed field last_updated . Before the feature, this field used to tell Async Manager that there the queue is ready for the new item to process. Now this decision is made based on queue_size < queue_size_limit condition.
• removed field queue_proc_msid. This field was never used in the code, and as now the same queue can have items processed by different management servers, we always refer to sync_queue_item.queue_proc_msid when need to figure out which management server is processing the job.
• removed queue_proc_time field. As before we could only process one queue item at at time, and after the change it can be more than one, this field was moved to sync_queue_item table.

sync_queue_item table

• new filed - queue_proc_time (got moved from sync_queue_table).

...

Code flow

...

Once the createSnapshot operation is requested either by the user via API, or by cloudStack snapshot scheduler, the code:

1) Determines the host information by a) retrieving the vm information from the volume. b) checking host_id for the vm. If not null, this would be the target host. If null, check last_host_id. If not null, the last_host_id is the targeted host. If both fields are null, then we don't do any synchronization - in most of the cases cloudStack won't experience it.

2) Once the host information is available, createSnapshot async job is scheduled and synced on the host object. If the number of current createSnapshot jobs processed against this host, is less than concurrent.snapshots.threshold.perhost allows, then the job is dequeued from the sync_queue_item table and sent to the backend.Once the job is dequeued, the sync_queue.counterqueue_size is incremented by one.

3) When the createSnapshot is purged from sync_queue_item table (means it's completed on the backend, and result is updated in async job table), the sync_queue.counterqueue_size field is decremented by one. And the next createSnapshot job standing in the queue for the same host, will start its execution.

4) If the createSnapshot job waits for more than createsnapshot.job.cancel.thresholdexpire.minutes for execution, then it gets expired and failure is returned to the API caller.

...