Introduction

KVM utilised under CloudStack uses the standard libvirt hook script behaviour: * https://libvirt.org/hooks.html

During install of the KVM CloudStack agent the script "/etc/libvirt/hooks/qemu" is populated based on the CloudStack provided file "libvirtqemuhook.in": * https://github.com/apache/cloudstack/blob/87ef8137534fa798101f65c6691fcf71513ac978/agent/bindir/libvirtqemuhook.in

This is a python script that carries out network management every time a Virtual Mavhine (VM) is started, stopped or migrated (as per the qemu script specification).

Other network configuration tasks can be done at the same time as the "/etc/libvirt/hooks/qemu" is called. Since the tasks in question are user-specific, they cannot be included in the CloudStack-provided script.

This feature allows other custom scripts to be called at the end of the "/etc/libvirt/hooks/qemu" execution.

Please note that CloudStack does not provide any scripts for other libvirt actions – and the scope of this feature request is limited to the "qemu" script itself.

Feature Specification Overview

  • This feature applies to KVM hosts. KVM utilised under CloudStack uses the standard libvirt hook script behaviour outlined in https://libvirt.org/hooks.html.

  • During install of the KVM CloudStack agent the script "/etc/libvirt/hooks/qemu" is populated based on the CloudStack-provided file "libvirtqemuhook.in". This is a python script that carries out network management tasks every time a virtual machine (VM) is started, stopped or migrated (as per the qemu script specification). 

  • Other network configuration tasks can be done at the same time as the "/etc/libvirt/hooks/qemu" is called. Since the tasks in question are user-specific, they cannot be included in the CloudStack-provided script.

  • The libvirt documentation page (https://libvirt.org/hooks.html#qemu describes the parameters that can be passed to the "qemu" script, based on what actions KVM and libvirt are carrying out on each VM: 'prepare', 'start', 'started', 'stopped', 'release', 'migrate', 'restore', 'reconnect', 'attach'.

  • The script contained in this feature allows for:
  1. Included scripts can be either a bash script and/or python.

  2. The script start-up and return commands are captured and logged.

  3. There are no limits to the number of scripts that can be included or called.

Acceptance Criteria

  • Should execute all scripts that meet the correct naming convention and that have been granted execute permissions to the underlying operating system.

  • Should log all actions.

  • Should timeout according to the "timeoutSeconds"global setting in the libvirtqemuhook.in script.

  • Should execute scripts alphabetically for each libvirt action.

  • All input arguments that are passed to the libvirt hook script must also be passed to each child script.

Usage

  • The cloudstack-agent package will install libvirtqemuhook.in script in the /etc/libvirt/hooks directory of libvirtd and rename the script to "qemu".

  • The documentation page: https://libvirt.org/hooks.html#arguments describes the arguments that can be passed to the "qemu" script. The input arguments are: 

  1. The first argument is the name of the object involved in the operation, or '-' if there is none. For example, the name of a guest being started.

  2. The second argument is the name of the operation being performed. For example, "start" if a guest is being started.

  3. The third argument is a sub-operation indication, or '-' if there is none.

  4. The last argument is an extra argument string, or '-' if there is none.

  • The operation argument is based on what actions KVM and libvirt are carrying out on each VM: 'prepare', 'start', 'started', 'stopped', 'release', 'migrate', 'restore', 'reconnect', 'attach':

  1. If an invalid operation argument is received, the hook script will log the fact, not execute any custom scripts and exit the program.

  2. All input arguments that are passed to the libvirt hook script are also be passed to each child script. 

  • For each of the above actions, the "qemu" hook script will find and run scripts by the name "<action>_<custom script name" in a custom include path /etc/libvirt/hooks/custom/. Custom scripts that do not follow this naming convention will be ignored and not be executed.

  • In addition to the libvirt standard actions, the hook script will also find and run scripts with "all" prefixed to the script name. For example: "all_<custom script name>". These scripts will run every time the hook script is called with a valid libvirt action.

  • In case of multiple scripts, they will be executed in sorted (alphabetical) order. The alphabetical ordering will use the file name part after the prefix and underscore has been removed from the file name. For example, if there are two files in the directory: all_cde.sh, migrate_abc.py; they will be sorted and executed in this order: migrate_abc.py, all_cde.sh.

  • Custom scripts can either be bash scripts and/or python scripts.

  • Custom scripts must be executable by the underlying host operating system. Non-executable scripts will be logged and ignored.

  • Each script's start-up and return commands will be captured and logged.

  • During execution of a custom script, the standard out (stdout) and standard error (stderr) outputs of the custom script will be logged (appended) to /var/log/libvirt/qemu-hook.log. If the custom script needs to log anything, it can also use this file for logging purposes.

  • There is a timeout setting in the libvirtqemuhook.in script that counts down at the start of every execution of a custom script. If the custom script is still executing after the timeout time has elapsed, the custom script will be gracefully terminated.

Configuration and usage

  • The timeout setting called "timeoutSeconds", at the top of the “qemu” script, has a default time out setting of 10 minutes, expressed as 10 * 60 seconds, and can be modified manually if a different time out is required.

  • For a custom script that needs to be executed at the end a specific action, create the script and prefix the script name with the action name, followed by an underscore. For example, if a custom script is named abc.sh then prefix 'migrate_' to the name to become migrate_abc.sh.

  • For a custom script that needs to be executed for any action, create the script and prefix the script name with the 'all' action, followed by an underscore. For example, if a custom script is named def.py then prefix 'all_' to the name to become all_def.py.

  • Grant each of these custom scripts with execute permissions so that the underlying host operating system can execute them.

  • Place these custom scripts in the custom include folder /etc/libvirt/hooks/custom/ so that the “qemu” script will find and execute them:

  1. In shell scripts include #!/bin/bash in the first line of the file.

  2. In Python scripts include #!/usr/bin/python in the first line of the file.

  • If a custom script needs to log anything, it can use the /var/log/libvirt/qemu-hook.log log file for logging purposes. Any standard out (stdout) and standard error (stderr) from the custom script will be logged by the “qemu” script.

  • No labels