...

Linux Containers (LXC) is a lightweight system virtualization that uses resource isolation instead of the hardware emulation approach used by KVM and Xen. For users who do not require full OS virtualization as provided by KVM and Xen, container technologies such as LXC provide an attractive performant solution for virtualization.

Cloudstack currently does not support containers such as LXC and OpenVZ.

Purpose

This docoument contains the design specification for LXC support in Cloudstack.

Status

Currently in development/testing, see last section below for detailsCode complete, feature has been merged to master branch.

References

• Jira: https://issues.apache.org/jira/browse/CLOUDSTACK-922^{Image Removed}
• Github: https://github.com/gilt/incubator-cloudstack^{Image Removed}
• Mail thread: http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.mbox/%3CCAD3R0XntF759eiafbLOdvftBQUX8yER2t20wTFGvhph6sgKWgw@mail.gmail.com%3E^{Image Removed}
• Meeting Notes:
  • 2013-02-05: http://wilderness.apache.org/archives/cloudstack-meeting-05_02_2013-11006.html^{Image Removed}

Feature Specification

...

We discussed the possibility of creating System VMs for LXC. There was concern with the complexity and potential issues involving iptables for the router inside an LXC container. As an intermediate solution we are going to use KVM System VMs inside the LXC Cluster.

Development/testing notes

For minimum requirements, you will need to run the management server and an LXC host. The LXC host has the exact same setup as KVM, the only difference is you specify the "lxc" hypervisor in the agent config file. The LXC host will run the system VMs under KVM.

I am currently running my dev/test setup on two machines: 1) runs the management and 2) the other runs the LXC agent. The following sections detail how to get the development code running so that you can spin up LXC containers with Cloudstack. It's not very elegant, but is the easiest way I've found to quickly get the system up with development code. There's probably an easier way to do this, like with RPMs, but I haven't gotten around to that yet.

1. Install Cloudstack 4.0

• Install the management server on machine 1
  • Skip loading the database
  • Do not import the System Template

• Install the agent
  • This will install the KVM agent, we will configure it later for LXC

2. Checkout and build latest code

• Checkout and build the code on both the management server and lxc host machines.
  • git clone git@github.com:gilt/incubator-cloudstack.git
  • cd incubator-cloudstack
  • mvn clean install

3. Update management server with 4.1 code

• Load the data model
  • cd incubator-cloustack
  • mvn -P developer -pl developer -Ddeploydb

• Run the management server
  • mvn -pl :cloud-client-ui jetty:run

• Load LXC data model
  • load cloudstack/setup/db/lxc.sql

...

Direct Networking

Libvirt supports direct attachment of the guest VM's network to a physical interface. To enable this mode, add the following to agent.properties:

libvirt.vif.driver=com.cloud.hypervisor.kvm.resource.DirectVifDriver
network.direct.source.mode=private (other values: bridge|vepa)
network.direct.device=eth0

NOTE: The network device that is specified should not be a slave to any bridges.

Environment setup and testing

Obtaining latest code

The LXC code has been merged to the master branch for Cloudstack:

git clone https://git-wip-us.apache.org/repos/asf/cloudstack.git

Follow directions in /docs/en-US/build-rpm.xml to build RPMs for Cloudstack.

Installing Cloudstack

I will not cover how to install Cloudstack, please use the latest online documents. There are a few things to note when using the LXC code:

1. Use the latest system VM images from Jenkins

Import the latest system VM image:

• http://jenkins.cloudstack.org/view/master/job/build-systemvm-master/

/web/cloudstack

...

/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary -u http://

...

jenkins.

...

cloudstack.

...

org/view/

...

master/

...

job/

...

build-systemvm-

...

master/lastSuccessfulBuild/artifact/tools/appliance/dist/systemvmtemplate-2013-04-14-master-kvm.qcow2.bz2 -h kvm -F

2. LXC container

Cloudstack will not come bundled with an LXC container image, so you will need to prepare one yourself or download one.

• Restart management server

4. Update LXC agent with 4.1 code

• Modify /etc/cloud/agent/agent.properties, add:
  • hypervisor.type=lxc

• Modify /etc/init.d/cloud-agent
  • Replace the "export CLASSPATH" line with

  • Code Block
    PCP="" for jarfile in <incubator-cloudstack-dir>/client/target/cloud-client-ui-4.1.0-SNAPSHOT/WEB-INF/lib/cloud-* /usr/share/java/jna.jar; do if [ ! -e "$jarfile" ] ; then continue ; fi PCP=$jarfile:$PCP done export CLASSPATH="$SCP:$DCP:$PCP:$JCP:/etc/cloud/agent:/usr/lib64/cloud/agent"

• Restart cloud-agent
  • service cloud-agent restart

5. Update Secondary Storage System VM with 4.1 code

In this step we bring up the system to the point where system VMs are created. After the Secondary Storage VM starts up, we will swap out the cloud jars with the ones we've built.

• From the Cloudstack UI: add a zone, pod, cluster, host, primary, and secondary for LXC
• After the Secondary Storage VM starts up, from the lxc host:

  • Code Block
    ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@169.254.x.x (SSVM local link ip) cp -r /usr/local/cloud/systemvm /usr/local/cloud/systemvm.orig rm /usr/local/cloud/systemvm/cloud-*.jar scp root@<lxc-host>:<incubator-cloudstack-dir>/client/target/cloud-client-ui-4.1.0-SNAPSHOT/WEB-INF/lib/cloud-*.jar /usr/local/cloud/systemvm service cloud stop service cloud start

6. Creating LXC instances

...