DUCC Project Documentation

Tutorials

Quick Start

Audience: DUCC Users/Administrators

Instructions on how to fetch, build and install DUCC 1.0 for single host system (my laptop!)
See DUCC Project Documentation for further details.

The following steps were performed on Red Hat Enterprise Linux Workstation release 6.4 (Santiago)

Prerequisites

Install svn
Install mvn
Install jdk (1.6 or better)
export JAVA_HOME=~/apps/java/java-home
Install latex
Install ant

Start sshd

[degenaro]$ sudo su root
[root]# chkconfig sshd on
[root]# /sbin/service sshd restart
[root]# exit
[degenaro]$

Create user ducc

[degenaro]$ sudo adduser ducc

Permit universal read+execute access to home directory for users degenaro and ducc

[degenaro]$ sudo su root
[root]# cd /home
[root]# chmod 755 degenaro
[root]# chmod 755 ducc
[root]# exit
[degenaro]$

Enable passwordless ssh for user ducc

 
[degenaro]$ sudo su ducc 
[ducc]$ cd
[ducc]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ducc/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ducc/.ssh/id_rsa.
Your public key has been saved in /home/ducc/.ssh/id_rsa.pub.
The key fingerprint is:
4f:4a:81:60:c9:a5:d8:88:80:f3:6c:a1:ef:31:8b:8e ducc@hal9000.apache.com
The key's randomart image is:
+--[ RSA 2048]----+
|o  .oo.          |
|+..=oo .         |
|.=o.o . .        |
|. +      .       |
| o      S .      |
|  +    . +       |
| o +    . .      |
|o o              |
|E.               |
+-----------------+
[ducc]$ cd .ssh
[ducc]$ cat id_rsa.pub >> authorized_keys2
[ducc]$ chmod 600 *
[ducc]$ exit
[degenaro]$

Insure JAVA_HOME is set, and PATH contains LaTeX

Sample code to set JAVA_HOME

.bashrc

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions

export JAVA_HOME=~/apps/java/java-home
export PATH=$PATH:/usr/local/tex

Fetch DUCC from svn

[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk ~/svn/ducc/trunk

Build DUCC jars

[degenaro]$ cd ~/svn/ducc/trunk
[degenaro]$ mvn clean install -Pbuild-duccdocs

Install

[degenaro]$ cd target
[degenaro]$ sudo su ducc
[ducc]$ tar -zxf uima-ducc-1.0.0-SNAPSHOT-bin.tar.gz -C ~ducc
[ducc]$ cd ~
[ducc]$ ln -s apache-uima-ducc-1.0.0-SNAPSHOT ducc_install 
[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./ducc_post_install

Check

[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./check_ducc -s
[ducc]$ exit
[degenaro]$

Optional: enable ducc_ling and login to Web-server via Linux authentication

[degenaro]$ sudo su root
[root]$ mkdir /local
[root]$ mkdir /local/ducc
[root]$ mkdir /local/ducc/bin
[root]$ chown ducc.ducc /local/ducc
[root]$ chown ducc.ducc /local/ducc/bin
[root]$ chmod 700 /local/ducc
[root]$ chmod 700 /local/ducc/bin
[root]$ cp ~ducc/ducc_install/duccling/src/ducc_ling /local/ducc/bin
[root]$ chown root.ducc /local/ducc/bin/ducc_ling
[root]$ chmod 4750 /local/ducc/bin/ducc_ling
[root]$ exit
[degenaro]$

[degenaro]$ sudo su ducc
[ducc]$ cd ~ducc/ducc_install/resources
1. edit ducc.properties
2. locate: ducc.ws.login.enabled = false
3. modify: ducc.ws.login.enabled = true
4. locate: ducc.agent.launcher.ducc_spawn_path=${DUCC_HOME}/admin/ducc_ling
5. modify: ducc.agent.launcher.ducc_spawn_path=/local/ducc/bin/ducc_ling
6. save

Start

[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./start_ducc -s
[ducc]$ exit
[degenaro]$

ducc-mon System Daemons

Start your favorite browser (firefox, chrome, opera) and visit the ducc-mon system daemons page.

http://localhost:42133/system.daemons.jsp

Submit a test Job

[degenaro]$ cd ~ducc/ducc_install/bin
[degenaro]$ export PATH=$PATH:$PWD
[degenaro]$ cd ~
[degenaro]$ mkdir ducc
[degenaro]$ cd ducc
[degenaro]$ cp -pR ~ducc/ducc_install/examples .
[degenaro]$ cd examples/simple
[degenaro]$ ducc_submit -f 1.job
Job 1 submitted
[degenaro]$

ducc-mon Jobs

Visit the ducc-mon jobs page.

http://localhost:42133/jobs.jsp

End.

Install WebServer Demo

Audience: DUCC Developers

Copy DUCC binary tar-ball to installation host

[degenaro]$ scp uima-ducc-1.0.0-bin.tar.gz degenaro@uima-ducc-vm.apache.org:~degenaro

Login to installation host

Note: Developers who are presently unable to login should request access via uima-dev mailing list

[degenaro@oc0883280806 Downloads]$ ssh degenaro@uima-ducc-vm.apache.org
...
degenaro@uima-ducc-vm:~$

Stop demo (if running)

degenaro@uima-ducc-vm:~$ killall -9 python

Stop ducc (if running)

degenaro@uima-ducc-vm:~$ cd ~/ducc/install/admin
degenaro@uima-ducc-vm:~/ducc/install/admin$ ./stop_ducc -a
...
.Stopping broker: localhost
.. FINISHED

Install new code such that ~/ducc/install is the $DUCC_HOME. For example, extract the downloaded zip to the versions folder...

degenaro@uima-ducc-vm:~/ducc/versions$ ls
apache-uima-ducc-1.0.0  apache-uima-ducc-1.0.0.rc10

...and update the symbolic link

degenaro@uima-ducc-vm:~/ducc$ ls -atl
total 12
drwxr-xr-x  3 degenaro degenaro 4096 Jan 16 16:32 .
drwxr-xr-x 18 degenaro degenaro 4096 Jan 16 16:04 ..
drwxr-xr-x  4 degenaro degenaro 4096 Jan 16 14:48 versions
lrwxrwxrwx  1 degenaro degenaro   32 Jan  7 18:48 install -> versions/apache-uima-ducc-1.0.0/

Run ducc_post_install

degenaro@uima-ducc-vm:~/ducc/install/admin$ ./ducc_post_install 
Python version:
2.7.3 (default, Sep 26 2013, 20:03:06) 
[GCC 4.6.3]
Ducc head is configured as uima-ducc-vm 

Initial /home/degenaro/ducc/versions/1.0.0/rc10/apache-uima-ducc-1.0.0/resources/ducc.nodes created.
Initial nodepool /home/degenaro/ducc/versions/1.0.0/rc10/apache-uima-ducc-1.0.0/resources/jobdriver.nodes created.

ActiveMQ is automanaged on node  tcp://uima-ducc-vm:61617?wireFormat.maxInactivityDuration=0&jms.useCompression=true&closeAsync=false
Initialized property "ducc.jvm" to /home/degenaro/local/sun/jdk1.7.0_45/bin/java
JAVA: java version "1.7.0_45"
JAVA: Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
JAVA: Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Java version: "1.7.0_45"
Java is verified.

WARNING
WARNING The default ssl pw of  quackquack  is configured. For security, change the property 
	ducc.ws.port.ssl.pw to a secure password in ducc.properties and rerun ducc_post_install.
WARNING

None
keystore created.

Web server keystore generated from ducc.properties

DUCC book installed into webserver root

cc -g    ducc_ling.c   -o ducc_ling
Initial ducc_ling is installed.  See the installation guide for multi-user setup.

Install local modifications

degenaro@uima-ducc-vm:~/ducc.local-mods$ ls admin
ducc.py
degenaro@uima-ducc-vm:~/ducc.local-mods$ cp -p admin/ducc.py ~/ducc/install/admin/
degenaro@uima-ducc-vm:~/ducc.local-mods$ ls resources/
ducc.properties
degenaro@uima-ducc-vm:~/ducc.local-mods$ cp -p resources/ducc.properties ~/ducc/install/resources/

Launch DUCC

degenaro@uima-ducc-vm:~/ducc/install/admin$ ./start_ducc -s
ENV: Java is configured as: /home/degenaro/usr/bin/java
...
uima-ducc-vm 
    Single user mode: bypassing ducc_ling checks.
    DUCC Agent started PID 10601
All threads returned

Create user logs directory

degenaro@uima-ducc-vm:~$ cd ~/ducc/user/logs/1.0.0
degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0$ mkdir rc10
degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0$ cd rc10
degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0/rc10$ mkdir logs
degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0/rc10$ cd ~/ducc
degenaro@uima-ducc-vm:~/ducc$ rm logs
degenaro@uima-ducc-vm:~/ducc$ ln -s user/logs/1.0.0/rc10/logs logs

Create and populate user examples directory

degenaro@uima-ducc-vm:~$ cd ~/ducc/user/examples/1.0.0
degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0$ mkdir rc10
degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0$ cd rc10
degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0/rc10$ mkdir examples
degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0/rc10$ cd ~/ducc
degenaro@uima-ducc-vm:~/ducc$ rm examples
degenaro@uima-ducc-vm:~/ducc$ ln -s user/examples/1.0.0/rc10/examples examples
degenaro@uima-ducc-vm:~$ cd ~/ducc/examples
degenaro@uima-ducc-vm:~/ducc/examples$ cp -pR ~/ducc/install/examples/* .

Submit a test Job

[degenaro]$ cd ~/ducc/install/bin
[degenaro]$ export PATH=$PATH:$PWD
[degenaro]$ cd ~/ducc/examples/simple
[degenaro]$ ducc_submit -f 1.job
Job 1 submitted
[degenaro]$

Visit uima-ducc-vm.apache.org:42133

Notes to run with simulated agents

cp -p ~/ducc.local-mods/resources/ducc.nodes ~/ducc/install/examples/systemtest/ducc.nodes
cd ~/ducc/install/examples/systemtest/
./start_sim -n ducc.nodes -c all
./reg_service -i 1

End.

Check out 1.0.0

[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/branches/uima-ducc-1.0.0

Child pages

DUCC

DUCC Project Documentation

Tutorials

Quick Start

Prerequisites

Start sshd

Create user ducc

Permit universal read+execute access to home directory for users degenaro and ducc

Enable passwordless ssh for user ducc

Insure JAVA_HOME is set, and PATH contains LaTeX

Fetch DUCC from svn

Build DUCC jars

Install

Check

Optional: enable ducc_ling and login to Web-server via Linux authentication

Start

ducc-mon System Daemons

Submit a test Job

ducc-mon Jobs

Install WebServer Demo

Copy DUCC binary tar-ball to installation host

Login to installation host

Stop demo (if running)

Stop ducc (if running)

Install new code such that ~/ducc/install is the $DUCC_HOME. For example, extract the downloaded zip to the versions folder...

...and update the symbolic link

Run ducc_post_install

Install local modifications

Launch DUCC

Create user logs directory

Create and populate user examples directory

Submit a test Job

Visit uima-ducc-vm.apache.org:42133

Notes to run with simulated agents

Check out 1.0.0