Page History

...

1.  Deploydb:
   - see if we can use http://code.google.com/p/flyway/ etc. for database upgrading/migration during development (suggested by John Burwell <jburwell@basho.com>)
2. Compile:
   - fix tomcat webapp compilation issue
3. UnittestsUnit tests:
   - fix unittests unit tests so they are more useful during compilation
4. Deploy and Debug:
   - configure tomcat plugin to do tomcat:deploy and tomcat:run, debug
5. Site/doc generation with maven

...

• cloud-iControl.jar http://zooi.widodh.nl/cloudstack/build-dep/cloud-iControl.jar
• cloud-manageontap.jar http://zooi.widodh.nl/cloudstack/build-dep/cloud-manageontap.jar
• vmware-vim.jar http://zooi.widodh.nl/cloudstack/build-dep/vmware-vim.jar
• vmware-vim25.jar http://zooi.widodh.nl/cloudstack/build-dep/vmware-vim25.jar
• vmware-apputils.jar http://zooi.widodh.nl/cloudstack/build-dep/vmware-apputils.jar

As of 4.3 you should no longer need to download the following netscaler jars. You may receive an error in the install-non-oss.sh script as it will still look for these files.

• cloud-netscaler.jar, cloud-netscaler-sdx.jar http://zooi.widodh.nl/cloudstack/build-dep/cloud-netscaler-jars.zip

...

...

Running a server for test/debug purposes.

On One can use maven to deploy and debug the management server. First export the MAVEN_OPTS variable to open a transport socket on port 8787 (same as before).

$ export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=500m -Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"

If you don't export using above configuration, the management server will run with no debugger attached.

This will run a jetty server on localhost port 8080 with the management server. Make sure that no other processes are using the port 8080. The default port for tomcat is 8080. You can either shut down tomcat service or switch to another port.

$ mvn -pl :cloud-client-ui jetty:run

To run a noredist management server:

$ mvn -pl :cloud-client-ui jetty:run -Dnoredist

NOTE : In case if BUILD process goes fine but, you are unable to access management server then please make sure that 8080 is opened in iptables.

To run awsapi app

$ mvn -Pawsapi -pl :cloud-awsapi jetty:run

Note: The following is a WIP tomcat-plugin feature, use the above jetty command for now.

$ mvn org.apache.tomcat.maven:tomcat7-maven-plugin:2.0:run -pl :cloud-client-ui -am -Pclient

Open the following URL on your browser to access the Management Server UI:
    http://localhost:8080/client/
Or,
    http://management-server-ip-address:8080/client

The default credentials are; user: admin, password: password and the domain
field should be left blank which is defaulted to the ROOT domain.

Integrated Simulator+Marvin test

The agent simulator and marvin are integrated into build steps to help a developer ensure that some simple tests pass before making a commit. The developer environment needs to have Marvin installed for the integration-test to work: These tests are lightweight and should ensure that your checkin doesn't break critical functionality for others working with branch: master.

To run the basic tests please refer to the Marvin wiki section on checkin tests.

Building Specific Versions of CloudStack

Checkout Building where we document building process for specific version of CloudStack such as 4.0 oss/nonoss etc.

Packaging

Building RPM packages

Use package.sh to build packages for linux systems that use rpms, like RedHat, CentOS and fedora.

$ cd packaging/centos63
$ ./package.sh

Building DEB packages

On Master:

$ mvn install -P deps && dpkg-buildpackage

WIP: Package using deb profile (feature under development uses jdeb plugin, in https://github.com/bhaisaab/incubator-cloudstack/tree/debs-maven debs-maven branch):

$ mvn package -P deb

Development with a IDE

Good Ol' Terminal

Following Building CloudStack section above.

IntelliJ IDEA

Import the project as Maven project. Build with maven, set socket type debugger on port 8787, follow Building CloudStack section above.

Eclipse

See Using Eclipse With CloudStack

Other Uses

RAT: ASF License Checking

$ mvn --projects='org.apache.cloudstack:cloudstack' org.apache.rat:apache-rat-plugin:0.10:check

The output file is in ${project.build.directory}/rat.txt, by default in target/rat.txt

Site Generation and Reporting

$ mvn site

Versioning / Releases

Update the version number of the project

mvn release:update-versions -DautoVersionSubmodules=true -Dnoredist -Pdeps,developer -DdevelopmentVersion=<version>-SNAPSHOT

Releases can also be done using the release plugin. This plugin will automagically take care of setting the currect version (removing the -SNAPSHOT), making a tag in SCM and deploying the jars to the maven repository. This should only be executed by the release engineer. Maven assumes that once a version is released it will never change anymore and released binaries will never be updated, so any subsequent releases will have to bump the version number.

Skip unit test

mvn install -DskipTests=true

Troubleshooting

• Offline Mode:

Set commandline parameter -o to force maven to only use the local repository. This might trigger build failures if artifacts are not available locally, but might speed up the build process in some cases.

$ mvn -o

• Out of memory error, out of heap space:

// Bash
export MAVEN_OPTS=-Xmx512m
// Windows
set MAVEN_OPTS=-Xmx512m

Install the SystemVM Template

The install-sys-tmplt script will need to be run (before you launch the management server). The example below is for XenServer, if you are using KVM or VMware use "-h" option to specify a different Hypervisor. The choice of the systemVM template from the download site is also Hypervisor specific. This script will place the systemVM image the secondary storage. The systemVMs (e.g. SecondaryStorageVM) are cloned from this image and with the systemv.iso that is built from the image the SystemVM is patched. The install-sys-tmplt script is meant to run in a production environment, and will fail if you run it from a development environment. Here are the step to run it from a development environment:

• Create /etc/cloudstack/management/ directory
• Copy your db.properties (or db.properties.override if you have it) to this location
• Run install-sys-tmplt script (if you have a previous systemVM image use -F to force update). You will find the script in your build directory as shown below:

...

For most versions of java on most version of macosx you need to add '-Djava.net.preferIPv4Stack=true' to that so to be on the save side; on macosx

$ export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=500m -Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n -Djava.net.preferIPv4Stack=true"

code-Djava.net.preferIPv4Stack=truecode

If you don't export using above configuration, the management server will run with no debugger attached.

On some (macosx) systems you might want to add "-Djava.net.preferIPv4Stack=true". Escpecially when using mixed wired and wireless networks.

This will run a jetty server on localhost port 8080 with the management server. Make sure that no other processes are using the port 8080. The default port for tomcat is 8080. You can either shut down tomcat service or switch to another port.

$ mvn -pl :cloud-client-ui jetty:run

To run a noredist management server:

$ mvn -pl :cloud-client-ui jetty:run -Dnoredist

NOTE : In case if BUILD process goes fine but, you are unable to access management server then please make sure that 8080 is opened in iptables.

To run awsapi app

$ mvn -Pawsapi -pl :cloud-awsapi jetty:run

Note: The following is a WIP tomcat-plugin feature, use the above jetty command for now.

$ mvn org.apache.tomcat.maven:tomcat7-maven-plugin:2.0:run -pl :cloud-client-ui -am -Pclient

Open the following URL on your browser to access the Management Server UI:
    http://localhost:8080/client/
Or,
    http://management-server-ip-address:8080/client

The default credentials are; user: admin, password: password and the domain
field should be left blank which is defaulted to the ROOT domain.

Integrated Simulator+Marvin test

The agent simulator and marvin are integrated into build steps to help a developer ensure that some simple tests pass before making a commit. The developer environment needs to have Marvin installed for the integration-test to work: These tests are lightweight and should ensure that your checkin doesn't break critical functionality for others working with branch: master.

To run the basic tests please refer to the Marvin wiki section on checkin tests.

Building Specific Versions of CloudStack

Checkout Building where we document building process for specific version of CloudStack such as 4.0 oss/nonoss etc.

Packaging

Building RPM packages

Use package.sh to build packages for linux systems that use rpms, like RedHat, CentOS and fedora.

$ cd packaging/centos63
$ ./package.sh

Building DEB packages

On Master:

$ mvn install -P deps && dpkg-buildpackage

WIP: Package using deb profile (feature under development uses jdeb plugin, in https://github.com/bhaisaab/incubator-cloudstack/tree/debs-maven debs-maven branch):

$ mvn package -P deb

Development with a IDE

Good Ol' Terminal

Following Building CloudStack section above.

IntelliJ IDEA

Import the project as Maven project. Build with maven, set socket type debugger on port 8787, follow Building CloudStack section above.

Eclipse

See Using Eclipse With CloudStack

Other Uses

RAT: ASF License Checking

$ mvn --projects='org.apache.cloudstack:cloudstack' org.apache.rat:apache-rat-plugin:0.10:check

The output file is in ${project.build.directory}/rat.txt, by default in target/rat.txt

Code Coverage and Quality checks

$ mvn -P developer,systemvm,quality -Dsimulator -Dnoredist clean install clover:aggregate clover:clover

The output coverage website is generated at target/site/clover/index.html

Site Generation and Reporting

$ mvn site

Versioning / Releases

Update the version number of the project

mvn release:update-versions -DautoVersionSubmodules=true -Dnoredist -Pdeps,developer -DdevelopmentVersion=<version>-SNAPSHOT

Releases can also be done using the release plugin. This plugin will automagically take care of setting the currect version (removing the -SNAPSHOT), making a tag in SCM and deploying the jars to the maven repository. This should only be executed by the release engineer. Maven assumes that once a version is released it will never change anymore and released binaries will never be updated, so any subsequent releases will have to bump the version number.

Skip unit test

mvn install -DskipTests=true

Troubleshooting

• Offline Mode:

Set commandline parameter -o to force maven to only use the local repository. This might trigger build failures if artifacts are not available locally, but might speed up the build process in some cases.

$ mvn -o

• Out of memory error, out of heap space:

// Bash
export MAVEN_OPTS=-Xmx512m
// Windows
set MAVEN_OPTS=-Xmx512m

Install the SystemVM Template

The install-sys-tmplt script will need to be run (before you launch the management server). The example below is for XenServer, if you are using KVM or VMware use "-h" option to specify a different Hypervisor. The choice of the systemVM template from the download site is also Hypervisor specific. This script will place the systemVM image the secondary storage. The systemVMs (e.g. SecondaryStorageVM) are cloned from this image and with the systemv.iso that is built from the image the SystemVM is patched. The install-sys-tmplt script is meant to run in a production environment, and will fail if you run it from a development environment. Here are the step to run it from a development environment:

• Create /etc/cloudstack/management/ directory
• Copy your db.properties (or db.properties.override if you have it) to this location
• Run install-sys-tmplt script (if you have a previous systemVM image use -F to force update). You will find the script in your build directory as shown below:

# mkdir -p /etc/cloudstack/management/
# cp ./client/target/cloud-client-ui-4.3.0-SNAPSHOT/WEB-INF/classes/db.properties /etc/cloudstack/management/
# ./client/target/generated-webapp/WEB-INF/classes/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary -u http://jenkins.buildacloud.org/view/All/job/build-systemvm64-master/lastSuccessfulBuild/artifact/tools/appliance/dist/systemvm64template-master-4.6.0-xen.vhd.bz2 -h xenserver -F

Remote debugging

To enable remote debugging on a system go to the cloudstack-management file (for centOS 7 this file can be found under the following path: /etc/default/cloudstack-management).|

Before CS 4.11 (tomcat7)

Add 

-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5000

 To the JAVA_OPTS:
JAVA_OPTS="-Djava.awt.headless=true -Dcom.sun.management.jmxremote=false -Xmx2g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m -Djavax.net.ssl.trustStore=/etc/cloudstack/management/cloud.jks -Djavax.net.ssl.trustStorePassword=vmops.com -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5000 "

Restart your management server (systemctl restart cloudstack-management).
You can now connect your java debugger on port 5000.

From CS 4.11 onwards (jetty):

Add 

JAVA_TOOL_OPTIONS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5000"

...