QuickCloud

CloudStack uses system vms extensively to offload the control plane ("management server") from being in the data path. There are 3 kinds of system vms:

Those used to provide network services such as DHCP, routing, firewall, load balancing (e.g., VirtualRouter)
Those used to provide image services ("Secondary Storage Virtual Machine" SSVM)
Those used to provide console services ("Console Proxy VM")

Some of the virtues of these system vms are:

automated scaling of services provided by these system vms
standardized deployments
minimal requirements on the control plane (i.e., just the hardware required to run the management cluster)

However the system vms are intertwined into the 'bootstrap' phase of the cloud and often times result in failure to start the cloud.
Another issue is that certain choices are made inside these system vms (for e.g., mount options for the secondary storage) that may not be ideal or optimal for the cloud environment.
A third issue is that the dependence on system vms hampers the introduction of support for new hypervisors.
Finally the requirement to bring up system vms prior to first user vm booting leads to longer development-debug cycles.
To avoid these issues while preserving the benefits of the system vms, it is proposed:

To enable a cloud to "boot" with services managed and running on non-cloudstack-managed servers
To enable a development environment based on DevCloud ("QuickCloud") that eschews the use of system vms
To allow the cloud administrator to choose to provide these services the "old" way after booting in the QuickCloud fashion

Secondary Storage

The secondary storage vm runs a java daemon that starts up on boot of the SSVM. This java daemon connects back to the management server. The daemon was originally designed to run alternately as a standalone process (just like the KVM agent), but over time the execution environment has been assumed to be the system vm. QuickCloud will bring back that ability

Console Proxy

The concerns and the approach are the same as Secondary Storage

DHCP and DNS

DHCP and DNS services are traditionally provided by the dnsmasq service running inside the system vm.

Basic Zone: It is proposed to run a single dnsmasq process in the entire zone that is controlled via a REST API
Advanced Zone: System VMs will continue to be used.

HOWTO: maven workflow for QuickCloud

[At present QuickCloud does not use any DHCP services]
1. Get the DevCloud2 VM up and running.
2. Mount the secondary storage export from the DevCloud2 on to your laptop:

export SS_MOUNT= mount-point-for-sec-storage e.g., /users/myhome/secondary-storage
mkdir $SS_MOUNT
mount -t nfs 192.168.56.10:/opt/storage/secondary $SS_MOUNT
mount -t nfs -o resvport 192.168.56.10:/opt/storage/secondary $SS_MOUNT #Mac OS X

3. Checkout the 'quickcloud' branch and build it and run it

git checkout quickcloud
mvn -P developer,systemvm  clean install -Dquickcloud
mvn -P developer -pl developer,tools/devcloud -Ddeploydb
mvn -pl :cloud-client-ui jetty:run

In a different terminal, execute the quickcloud zone creation script via marvin

mvn -P developer -pl tools/devcloud -Dquickcloud

4. Open two more terminals. In the first terminal

 cd services/secondary-storage

In the second terminal

 cd services/console-proxy/server

5. In the secondary-storage terminal

echo "mount.path=/users/myhome/secondary-storage" >> conf/agent.properties
mvn exec:java

In the console-proxy terminal,

 
echo "paths.pid=." > conf/environment.properties
echo "developer=true" >> conf/agent.properties
mvn exec:java -Pquickcloud

6. Open the GUI and deploy a VM. Although the VM won't get an IP, you should be able to view the console and log in and set an ip.

HOWTO: Non-maven workflow for QuickCloud

[At present QuickCloud does not use any DHCP services]
0. The following assumes you are working on the same server as the management server, but equivalently you can set up another VM/server for the purpose of running these system services. This will also work with DevCloud2.
1. Mount the secondary storage export from your NFS server on to a mount point on your (management) server

export SS_MOUNT= mount-point-for-sec-storage e.g., /var/mount/secondary-storage
mount -t nfs $NFS_IP:$NFS_EXPORT $SS_MOUNT #use your favorite NFS mount options

2. Find the 'systemvm.zip' archive on your management server

find /usr/share -name systemvm.zip #centos
find client -name systemvm.zip #in your git workspace

3. Create 2 directories for running the SS and CP services respectively. For example:

mkdir -p /var/cloudstack/secondary #or wherever
mkdir -p /var/cloudstack/console
cp systemvm.zip /var/cloudstack/secondary
cp systemvm.zip /var/cloudstack/console

4. Secondary Storage server:

 cd /var/cloudstack/secondary
unzip systemvm.zip

Edit conf/agent.properties and add

zone=1 #or actual zone id from management server database table datacenter
host=<ip of management server>
guid=<some guid> #use uuidgen to generate a guid
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource 
#resource=com.cloud.storage.resource.PremiumSecondaryStorageResource for VMWare
mount.path=<value of $SS_MOUNT>

Run the secondary storage service:

./secstorage.sh

5. Console Proxy Server:

 cd /var/cloudstack/console
unzip systemvm.zip