Userdata as a first class resource

Introduction

Currently, userdata content is directly provided during VM deployment and updating a VM. Userdata can be made a first class resource much like the existing SSH keys because it will become easy to manage userdata as there are many usages of it in creating VMs, eg. cloud-init.

Purpose

To make userdata as a first class resource.

Requirement

To introduce userdata as a first class resource much like SSH keys where it can be registered/listed/removed and be linked to VMs and templates.

Document History

Feature Specifications

In this section, we define the scope and high-level specifications for the feature which is as follows:

1. Userdata can be registered/listed/removed in CloudStack using the new APIs provided. 
2. While deploying a VM, a user can select existing Userdata resource UUID with (optional) custom key-value parameter map. The key-value parameter map is to provide the values for the variable keys (if any) declared in the userdata content. 
3. A new API to reset/update UserData for an already deployed VM, similar to the reset SSH key API/feature. The reset/update can be done using updateVirtualMachine or resetUserDataForVirtualMachine API.  
4. UserData can be linked/mapped to a template both on registration/upload/editing of a template using linkUserDataToTemplate API. The same API can be used to unlink the mapping of userdata and template. 
5. UserData linked to a template should specify the UserData link policy: 
   1. Allow Override: Allow users to override UserData for the template during VM deployment or on reset. This is the default override policy if not specified 
   2. Deny Override: Override of UserData isn’t allowed during VM deployment or on reset. 
   3. Append Only: Don’t allow users to override linked UserData but allow users to pass userdata content or ID that should be appended to the linked UserData of the template. 
6. Passing both registered UserData UUID and existing API parameter for userdata content is not allowed.  

UserData can be registered with custom userdata content having variables declared in it. While deploying a VM using that registered UserData with variables, users have to pass the custom key value parameter map to fill the variables in the userdata with the values provided in the map. 

Custom UserData details that are passed as key/value map details are saved as meta-data files in both config drive and VR, which in turn could support jinja based instance meta-data feature of cloud-init, refer to https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html. 

For example, if a user data has a variable key1 defined as “{{ ds.meta_data.key1 }}” and user provided the custom key value pair as {key1=value1} then the variable will be rendered as "value1". 

Design & Implementation

API Changes

Following are the new and exciting API changes made to support this.

New APIs added:

Existing API changes:

Schema changes:

A new table “user_data” will be created with following columns

• `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT ‘id’,
• `uuid` varchar(40) DEFAULT NULL,
• `account_id` bigint(20) unsigned NOT NULL COMMENT ‘owner, foreign key to account table’,
• `domain_id` bigint(20) unsigned NOT NULL COMMENT ‘domain, foreign key to domain table’,
• `name` varchar(256) NOT NULL COMMENT 'name of the user data',
• `user_data` mediumtext COMMENT ‘value of the userdata’,
• `params ` mediumtext COMMENT ‘value of the comma-separated list of parameters’,

Existing table “user_vm” will have following changes

• New column `user_data_id` will be created which has foreign key constraint with `id` in `user_data` table.
• New column `user_data_details` will be created which holds the key value pair map for the custom user data provided from the APIs

Existing table “vm_template” will have following new columns:

· `user_data_id` with foreign key constraint with `id` in `user_data`

· `user_data_link_policy` (to store enum/string value, default none/null is interpreted as override allowed)

Service Layer and VR Changes

The service layer changes are largely coupled with the implementation of the APIs. VM and Template related manager classes to support relevant APIs that are described in the sections above.

Depending on the UserData link policy, the service layer must allow override, deny override or append user provider userdata or userdataid to process and create the effective UserData that must be provisioned to the VM via config drive, VR Userdata providers are defined in the network’s network offering to which the VM is deployed to

To support custom user data details, specific changes are required in both userdata and metadata providers supported and implemented for config drive and VR to support userdatadetails key/value map that is passed during VM deployment or VM update (for updating linked userdata/userdataid and userdatadetails).

To support Jinja based cloud-init userdata, the passed userdatadetails parameters key/value pair must be saved to specific file(s) in the config drive or for the VR to support cloud-init instance metadata feature (Refer to https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html).

When cloud-init based UserData begins with “## template:jinja” cloud-init will use jinja to render the file and expand variables defined it in using the instance metadata custom key/value pair that is available to cloud-init. Cloud-init produces a simple json object in /run/cloud-init/instance-data.json and instance-data-sensitive.json which represents a standardized and versioned representation of the metadata it consumes during initial boot. Any instance-data-sensitive.json variables are surfaced as jinja template variables. The JSON file contains metadata from the cloud provider datastore. The values can be used to render the jinja template.

The key/value pairs are passed as userdatadetails will be saved in VR as extra meta data. For example, VR has a new file /var/www/html/metadata/<VM IP>/<key1> with content “value1".

When VM is started, in the `/run/cloud-init/instance-data.json` file it has a new line in `meta_data` block:

"meta_data": { ...

"instance-id": "42f3c309-72da-4851-9d00-46c627655b7c",

"key1": "value1",

"local-hostname": "admin-001",

... },

If VM has user data in jinja template, “{{ ds.meta_data.key1 }}” will be rendered as "value1".

UI Changes

• Following are UI changes made to support the userdata to register/list/delete, in deploy/update/reset virtual machine and to link to templates. 

  • A new sub-section under “Compute” tab is added for userdata, where userdata can be registered, listed and removed. 

  

  • Register userdata form 

   

  • Register userdata form with custom parameters 

   

  • To support linking of userdata to template/ISO during registration, upload and edit 

   

  • To support unlinking of userdata to template/ISO during edit using empty values 

   

  • In “Add Instance” form, under “Advanced Mode”, there is a userdata section to either select a registered userdata or a userdata content 

   

  • If a userdata is already linked to the template which is being used in VM deployment, following options are show based on the userdata override policy defined during linking of userdata and template 

   

  A text is shown which describes the userdata linked to the template and the override policy and based on the override policy we get the option to either override or append the userdata. If the override policy is deny, then we don’t provide option to select or provide userdata. 

  If the userdata linked to template or the selected userdata has the custom parameters declared, then we provide a table to provide key value pairs. 

  • A reset userdata button is added for stopped VM, next to reset ssh key button 

   

  • Reset userdata form looks like below, this is similar to the userdata section in “Add Instance” form