Page History

...

These certificates can be in turn be used to configure TLS for libvirtd while adding a new KVM host using the cloudstack-setup-agent script that already configures libvirtd.conf with following: 

listen_tcp=1
tcp_port="16509"
auth_tcp="none"
listen_tls=0

Such additional configuration will be bound by a (cluster scope) global setting "kvm.libvirtd.tls.enabled", when set to true will perform the following during host addition via the cloudstack-setup-agent script by passing an additiona flag such as "-s" or "--securelibvirtd":

• In the /etc/libvirt/libvirtd.conf (and similar on other non-centos distros) file, the paths to CA, server/client, key files are already assumed to perform minimal changes to the config file we can create some symlinks if the source files exist at /etc/cloudstack/agent path:

ln -s /etc/cloudstack/agent/cloud.ca.crt /etc/pki/CA/cacert.pem
mkdir -p /etc/pki/libvirt/private
ln -s /etc/cloudstack/agent/cloud.crt /etc/pki/libvirt/clientcert.pem
ln -s /etc/cloudstack/agent/cloud.crt /etc/pki/libvirt/servercert.pem
ln -s /etc/cloudstack/agent/cloud.key /etc/pki/libvirt/private/clientkey.pem
ln -s /etc/cloudstack/agent/cloud.key /etc/pki/libvirt/private/serverkey.pem

...

• In addition to these changes, remove "listen_tls=0" set by cloudstack-setup-agent. Also add: set authTLS certificate based auth will be enabled therefore explicit authentication using something sasl is not necessary but may be done out-of-band by an admin, by default we'll limit settings to: auth_tcp="none" (this is the current default) and auth_tls="none" (this will be added if global setting was true) in libvirtd.conf.
• Port 16514 needs to be allowed/enabled by firewall/iptables rules, suitable changes will be done in cloudstack-setup-agent and other relevent files.

Proposed Changes

Changes in (KVM) agent:

• Add a new flag such as "-s" in cloudstack-setup-agent that automates the symlinking and additional setup of TLS in libvirtd.conf. The other requirement is that the source cert/pem files already exist at /etc/cloudstack/agent.
• Libvirtd-tls secured hosts should write a config such as libvirtd.tls.enabled=true in agent.properties file, this will be used by the agent to tell management server that it can support qemu+tls:// during migration.
• When certificates are renewed keystore-cert-import may be passed a flag to automate the above (wrt configuring libvirtd and reload/restarting it)

...

• Introduce a new global setting "kvm.libvirtd.tls.enabled" with Scope.Cluster to enable TLS for KVM hosts (for live migration etc), this will be disabled enabled by default. However, when this setting is enabled for a cluster or globally, new hosts when added will call We chose to enable this by default as CA framework generates certs during addition of new hosts using the addHost API, and the cloudstack-setup-agent with a "-s" flag will enable TLS in libvirtd.nonetheless check that the certificates actually exists before configuring libvirtd.conf. This setting can also be disabled for a cluster or globally. This will be mentioned in the release notes.
• Changes in keystore-cert-import will be suitably made to additionally reload/restart libvirtd on certificate changes (we need to check if this causes any issues, when agent is already running – this may be done via the agent when its internal task queue is 0)
• When a KVM agent connects to the management server, it will share its host details whether it has its libvirtd TLS setting enabled by using StartupRoutingCommandvia the StartupRoutingCommand. And this information will be persisted in host_details.
• When a VM migration is requested, the MigrateCommand will contain information about the destination host whether it is TLS enabled or not and if kvm.libvirtd.tls.enabled set to true the tls enabled setting for source and destination hosts which this will cause the MigrateCommand handler to include the URI to use for migration, i.e. "qemu+tls://" if both source and destination hosts have libvirtd with TLS enabled, otherwise for backward compatibility qemu+tcp:// will be used (a warning may be logged when qemu+tcp:// will be used).
• VM migration can fail (with errors such as CA.pem not found etc) when one of the hosts is not TLS enabled .

New CloudStack environments:

• New CloudStack environments will have kvm.libvirtd.tls.enabled set to true, and have TLS enabled libvirtdbut in agent.properties it says it is (in case of any out-of-band changes) such an error will be logged and may be exposed via the migrateVirtualMachine API.

Upgraded/mixed CloudStack environments:

• After upgraded, existing KVM hosts may not have CA-frameworks and certificates generated and will have kvmnot have the kvm.libvirtd.tls.enabled set define and will assume this to be false.
• Post-upgrade, newly added KVM hosts will have libvirtd configured to use TLS if as the global setting is will be true (globally or at the host's cluster level)
• On-boarding strategy can be:
  • Set Keep the global setting as true to enable tls on kvm/libvirtd for the cluster or globally
  • Execute provisionCertificate for API and provide the host id for all hosts and this will create certificates as well as configure libvirtd/tls (if the global setting is enabled)

Outstanding questions:

• The use of global settings can be removed, if we decide that new hosts will have TLS enabled, and also given that CA framework creates/provisions the certs.
• We can further simplify the design and setup TLS for libvirtd if we decide that adding of hosts and provisioning of certificates on host will always configure and enable tls for libvirtd.
• Should automation of libvirtd+tls configuration be automatic given we've a CA framework now, or give flexibility to people for any out-of-band changes/configuration, and toggle this automation with a global setting (kvm.libvirtd.tls.enabled).
• We may further removkvm.libvirtd.tls.enablede and simplify action items in this FS.  
  • via the keystore-cert-import script.

References

https://wiki.libvirt.org/page/TLSSetup

...