Bug Reference

CLOUDSTACK-....

Branch

template-upload branch off master

Introduction

To register a template/upload volume, one needs to host it some where as cloudstack only accepts a url. This eliminates that and enables the user to quickly upload a template/volume to cloudstack.

use cases

1. Anyone who wants to quickly upload a template/volume and test cloudstack without having the need to host it.

Design

1. Browser based upload
   1. Management Server generates a unique pre-shared key (PSK1) on install and shares it with the SSVM agent. Nobody else has access to this key of course.
   2. Browser UI requests the upload form
   3. MS sends down a form with standard fields pre-populated such as the upload url, url expiry time, account uuid, content-type AND a signature calculated over these fields. The signature is calculated using the pre-shared key PSK1. The user can be asked for inputs such as the filename on her local computer, format, hypervisor and the MD5 sum of the file. The form action is to POST to the SSVM's public IP
   4. The browser POSTS the form as enctype="multipart/form-data" over HTTPS to the SSVM's public IP

   5. The Apache web server on the SSVM matches the url, decrypts and passes on to the SSVM java agent as plain HTTP. The SSVM agent checks the signature against the form fields using the key PSK1. Once the fields are verified, it writes the  to the NFS storage to the location encoded in the URL. 

   6. Once the upload completes, the MD5 checksum is compared (if available from step c). If it matches, the agent informs the MS that the upload is successful and the template is registered. The MS returns the template identifier to the agent and the agent returns this id
2. API-based upload
   1. Preshared key as before
   2. New API request "GetUploadParams" encoded the usual CloudStack API way using api keys. Request parameters include format, hypervisor, md5 sum
   3. MS response to (b) is a json document which includes an inner json structure detailing the form parameters as in (c) above. Outer json contains the upload URL and the pre-calculated signature (using PSK1) over the inner json.
   4. API user makes a PUT request over HTTPS to the URL obtained in (c)
   5. as before
   6. as before.

Sequence Diagram

1. failure at any point during the form upload to ssvm apache would return error to the user and DOWNLOAD_ERROR when the management server pings for download status.

API changes

1. new API GetUploadParams which takes all the params as the registertemplate/uploadvolume except for URL and  which returns all the required params plus a post URL to be used to uploadVolume/registerTemplate
   
   A sample request and response would look below
   request params: 

   name=lxc1
   displayText=lxc1
   zoneid=-1
   format=TAR
   isextractable=false
   passwordEnabled=false
   isdynamicallyscalable=false
   osTypeId=1b510c30-3352-11e4-aaca-a5c7f57670d0
   hypervisor=LXC
   requireshvm=false

   
   response params: { "postURL":"https://ssvmpublicip/upload/key",params:{"store":{"com.cloud.agent.api.to.NfsTO":"nfs://10.147.28.7/export/home/rajani/lxc/secondary"}, "hvm":false,"description":"lxc1","maxDownloadSizeInBytes":53687091200,"id":207,"resourceType":"TEMPLATE","installPath":"template/tmpl/2/207", "format":"TAR","accountId":2,"name":"207-2-09c78adc-065f-3dc8-9621-9e7a036d7dc9"}}

UI Changes

1. A new register template and upload volume UI which should be able to accept "browse and upload" file. 

DB Changes

No db changes.

Major Tasks

1. setting up https on SSVM, opening the relevant ports doing the SSL handshake
   1. http://www.symantec.com/connect/articles/apache-2-ssltls-step-step-part-1
2. initial keystore exchange and authentication using the keystore
3. understanding template code 

Future

1. Download progress to the user (http://creativejs.com/tutorials/advanced-uploading-techniques-part-1/ )
2. split upload and join in apache so as to not hit browser timeouts