Bug Reference

CLOUDSTACK-6122

Introduction

Purpose

LXC support was added to CloudStack in 4.2 release. FS : LXC Support in Cloudstack. The objective of this feature is to enhance LXC support by adding more functionality. This document contains design specification of the enhancements.

References

LXC Support in Cloudstack

CLOUDSTACK-922

Feature Specifications

LXC was implemented as hypervisor in cloudstack. Following operations are currently supported:

• Add an LXC cluster 
• Add an LXC host 
• Register an LXC template 
• Spin up an LXC container

LXC containers are created using libvirt in cloudstack. Libvirt domain xml will be modified to support additional features. KVM system Vms are used for LXC, so most of the networking features are already supported.

Storage

• ROOT Disk: NFS and local storage is supported for ROOT Disk
• DATA disk: Support data disk while creating container. Only Ceph based data disk will be supported
• Support attach/detach disk. attach-disk and detach-disk are supported by libvirt (since 1.0.2) for lxc. Same will be leveraged to support attach/detach data volume.
• Support Ceph storage that will appear as block device on host using RBD

  • Configure and create Ceph RBD on host and make it visible in the container using host passthrough configuration

  • <hostdev mode='capabilities' type='storage'>
       <source>
          <block>/dev/rbd1</block>
        </source>
    </hostdev>

• Support for other block devices: Any block device if made available on the host, can be made available to the container also. Currently only Ceph RBD is supported.

Ceph RBD

• Libvirt packages with RDB support have to be installed on host
• Enable ceph repo to install rbd dependencies

[ceph-noarch]

name=Ceph noarch packages

baseurl=http://ceph.com/rpm-{ceph-release}/{distro}/noarch 

• When RBD storage pool is added as primary storage following xml is sent for pool creation on host

<pool type='rbd'>
<name>ae84381b-c273-3c8e-925f-8e384d3216b4</name>
<uuid>ae84381b-c273-3c8e-925f-8e384d3216b4</uuid>
<source>
<host name='10.147.59.89' port='6789'/> - Monitor host
<name>rbd</name> - Pool name
<auth username='admin' type='ceph'>
<secret uuid='ae84381b-c273-3c8e-925f-8e384d3216b4'/> - secret containing auth key
</auth>
</source>
</pool>

• Secret contains authentication details. Secret is created when RBD pool is created

<secret ephemeral='no' private='no'>
<uuid>ae84381b-c273-3c8e-925f-8e384d3216b4</uuid>
<usage type='ceph'>
<name>admin@10.147.59.89:6789/rbd</name>
</usage>
</secret>

• When Cloudstack created rbd Data disks, rbd images are created in the pool
• Rbd image is mappped on to host using "rbd map" command
• Mapped rbd device on host is then made available to container during attach disk

Template

Registering a template and  Launching container from template is already supported. Additional functionality:

• Create template from ROOT volume. Template is a compressed tar.gz of the root file system. Create template from volume will copy the tar.gz of the root file system to secondary storage

Networking

 KVM system Vms are used and both VPC and non-VPC are supported. Networking services Source NAT, DNS, PF, LB, Vpn, Firewall, Security Groups, Network ACL are supported. Additionally nic hot-plug to userVm (container) is also supported.  Secondary IPs are also supported.

Console Access

CLOUDSTACK-4990

Below libvirt doesn't seem to work for LXC. autoport option did not assign a port.

<graphics type="vnc" port="-1" autoport="yes" listen="10.147.28.11" keymap="en-us"/>

Database

No changes

New APIs

None

Limitations

• No Console access
• No Live migration
• No migration across clusters
• No snapshot support
• No upload/download volume support
• No template creation from ROOT volume
• No ISO support for create Vm

Additional steps 

Following steps need to be performed before adding LXC host to management server

• Fix cgroup co-mounting error in RHEL7: 
  • When cpu,cpuacct are co-mounted, libvirt fails to create containers. This issue is fixed in later versions of libvirt. For libvirt 1.1.1 which comes with RHEL7, below work-around is required
  • Rhel7 Initial RAM disk (initrd) uses systemd. Any changes made to cgroup will be lost after reboot.  So rebuild the initrd

  • Set JoinControllers to empty string in /etc/systemd/system.conf. Empty string ensures that systemd mounts all controllers in separate hierarchies

JoinControllers=

• • Backup initrd.

cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak

• • Rebuild initrd

dracut -f

• • Reboot host

• • ToDo: cloudstack-setup-agent script should check if cgroup is properly mounted and raise exception if cgroups are not properly configured - CLOUDSTACK-7524

Versions

LXC version 1.0

Libvirt Version 1.2.1

Open Issues

LXC hosts need not be hvm enabled. So, a VM can also act as a host for LXC conatiners. Currently, cloudstack doesn't allow hosts which are not hvm enabled. This check has to be relaxed for LXC.