Page History

...

The above diagram is a bird’s eye view of the overall architecture of the “agents framework” that shows end-to-end integration, communication and orchestration of services/solutions/tools. The agents framework consists of four main components: (1) a bootstrapping agent that runs in a user VMs and securely communicates with a (2) provisioning server that runs in a VR, the provisioning server gets recipes and commands from a (3) agents orchestration manager in management server’s resource layer, each integration or feature such as reset password will have it’s own (4) API or backend plugin that gets triggered by a corresponding CloudStack UI component or plugin or via a CLI such as CloudMonkey.By having a backend plugin that not only provides a high level provisioning API

To provide a better user experience via CloudStack’s UI, we can enhance the existing CloudStack UI plugin framework by making suitable changes in the CloudStack UI code so that custom UI plugins can be written that can add buttons on a resource (for example, reset password button on a VM page) or add tabs with custom content in resource wizards (for example, selection options of monitoring or backup solution when deploying a VM using the deploy VM wizard). The UI components can also trigger notifications from CloudStack regarding VR health and status of the operation/solution being deployed to the user VM (such as success/failure of resetting user VM password, or setting up monitoring tool in user VM), and such events can also be published on CloudStack event bus (for example, on RabbitMQ). A backend plugin can provide high level provisioning APIs (to both CloudStack UI and CLIs via APIs)

...

and this way we can have feature specific implementation decoupled from core CloudStack codebase. The plugin would provide an asynchronous push mechanism to provision software tools/services/solutions inside user VMs. The provisioning server itself could be based on something lightweight such as Ansible, Chef, Puppet or something that we customize or write from scratch. We propose to write a lightweight-bootstrapping agent that is cross-platform (works on Linux, Windows etc) and standalone (has no external library dependency) binary (both 32 and 64 bit) that is pre-seeded on user VM templates. This approach is similar how reset password feature works as it too needs a reset password script to be installed on a user VM template. A general bootstrapping agent will ensures that we have a single agent that works with any service whether it’s reset password feature or something else; and when a user VM starts with such a bootstrapping agent we’ll have end-to-end communication - from the CloudStack UI to the CloudStack backend plugin (using its asynchronous/non-blocking APIs) to CloudStack’s resource layer to the agent-provisioning server running in the Virtual Router to finally the bootstrapping agent that is running in the user VM. The bootstrapping-agent has the role to get commands, recipes or configuration rules from the agent-provisioning server and execute them.

In case a user uploads a custom template and starts VM, it will have no bootstrapping agent. In this case, the backend plugin retries provisioning couple of times (retries can be configured by a globally configuration) and when it gives up it can alert or email the user to take suitable steps to setup a bootstrapping-agent on the VM. An alternative approach to failure handling could be to attach an ISO to the user VM containing the bootstrapping agent that allows users to easily install the agent.

Security Aspects

To provision software into user VMs, the bootstrapping-agent needs to run in a privileged mode. For example, if a tool/service/solution needs to be installed or a password needs to be reset, the bootstrapping-agent needs to run as root/admin but if root/admin level access is not required for provisioning operations we would not want to run it in privileged mode. This would be configurable when pre-seeding the agent on a VM template, or when doing manual installs or patching using an agent installer ISO. We aim to implement a bootstrapping agent library that can be used to implement custom bootstrapping agents (for example, a bootstrapping agent which restricts what kinds of receipes/operations can run on a user VM, or another bootstrapping agent that allows for any recipes/commands like a Puppet/Chef client). Implementation will aim to use SSL/TLS whenever communicating with non-local services, for example between VR and user VMs.