...

• feature shall support upload of template through https POST request
• feature shall support upload of volume through https POST request
• uploading same volume or template shall be treated as new upload by management server
• failure to upload volume or template is irrecoverable, it needs to be reinitiated by requesting a new upload url
• management server (or SSVM agent ?) shall ensure cleanup of partially uploaded volume or template in case of failure (?)
• management server shall be able to give status (percentage, bytes etc) of the upload using the list templates/volume api call
• a global param to limit the max upload size (is there an existing one that can be reused?)size 
• management server will use enforce the current resource limits for templates/volumes and would error on reaching the limits. 
• This is hypervisor agnostic and should work for all hypervisors SSVM apache will be able to handle parallel upload and will be limited by the number of connections in the apache settings

...

Out-of-Scope

1. No unencrypted or http version of this the POST will not be available

2. Upload progress to the user when the POST request is active  (http://creativejs.com/tutorials/advanced-uploading-techniques-part-1/ )

3. split upload and join in apache so as to not hit browser timeouts

4. post url should be active only for one time call - return proper error message on subsequent calls - how do we determine if its already used?

5. Admin to list all the uploads happening and cancel any if required

6. Multiple files for the same template/volume will not be supported for nownot be supported for now

Limitations

1. post url should be active only for one time call - Its partially handled now. The limitation needs to be addressed.

Design

process flow

The browser-based uploading process will have four main steps:

...

1. Management Server generates a unique pre-shared key (PSK) and shares it with the SSVM agent. 
2. New API request "getUploadParams" encoded the usual CloudStack API way using api keys. Request parameters include format, hypervisor, md5 sum etc.
3. Management Server response to (b) returns post url to upload the file, params to send to the url while uploading the file and a SHA1 signature generated using PSK and params json.  
4. User should upload the file over https POST request to the url obtained in (c). The signature and params returned in (c) should be passed as is.
5. The Apache web server on the SSVM matches the url, decrypts and passes on to the SSVM java agent as plain HTTP. The SSVM agent checks the signature against the params using the key PSK and SHA1. Once the params are verified, it writes to the NFS storage to the location encoded in the URL. 
6. Once the upload completes, the MD5 checksum is compared (if available from step c). 
   

...

1. c). 

pre-shared key management

...

The limitation here is that if the template/volume sync cleanup happened and the template is cleaned, agent would start accepting the url again. This will be fixed later. 

...

The NFS Store url, path and some other information are exposed to the end user in the response to getUploadParams. This is done so that agent gets all the information required to download the template with the POST request.

performance

To increase the number of parallel uploads, multiple SSVMs needs to be created. This will be handled automatically by the management server using the system.vm.auto.reserve.capacity, secstorage.capacity.standby and secstorage.session.max configuration variables.

UI Changes

A new register template and upload volume UI which should be able to accept "browse and upload" file. 

...