DUCC

[degenaro@oc4534203605 ~]$ env | grep JAVA_HOME
JAVA_HOME=/home/degenaro/apps/java/java-home
[degenaro@oc4534203605 ~]$ /home/degenaro/apps/java/java-home/bin/java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470sr9-20150417_01(SR9))
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 Compressed References 20150406_242981 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR9_20150406_1443_B242981
JIT  - tr.r11_20150401_88894
GC   - R26_Java726_SR9_20150406_1443_B242981_CMPRSS
J9CL - 20150406_242981)
JCL - 20150414_02 based on Oracle 7u79-b14
[degenaro@oc4534203605 ~]$ 

[degenaro@oc4534203605 ~]$ sudo su root
[sudo] password for degenaro: 
[root@oc4534203605 degenaro]# chkconfig sshd on
[root@oc4534203605 degenaro]# /sbin/service sshd restart
Stopping sshd:                                             [FAILED]
Starting sshd:                                             [  OK  ]
[root@oc4534203605 degenaro]# exit
exit
[degenaro@oc4534203605 ~]$ 

[degenaro@oc4534203605 ~]$ sudo adduser ducc

[degenaro@oc4534203605 ~]$ sudo su - ducc
[ducc@oc4534203605 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ducc/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ducc/.ssh/id_rsa.
Your public key has been saved in /home/ducc/.ssh/id_rsa.pub.
The key fingerprint is:
4f:4a:81:60:c9:a5:d8:88:80:f3:6c:a1:ef:31:8b:8e ducc@hal9000.apache.com
The key's randomart image is:
+--[ RSA 2048]----+
|o  .oo.          |
|+..=oo .         |
|.=o.o . .        |
|. +      .       |
| o      S .      |
|  +    . +       |
| o +    . .      |
|o o              |
|E.               |
+-----------------+
[ducc@oc4534203605 ~]$ cd .ssh
[ducc@oc4534203605 .ssh]$ cat id_rsa.pub >> authorized_keys2
[ducc@oc4534203605 .ssh]$ chmod 600 *

[ducc@oc4534203605 .ssh]$ cd
[ducc@oc4534203605 ~]$ pwd
/home/ducc
[ducc@oc4534203605 ~]$ ls
Desktop  Downloads  uima-ducc-2.0.0-bin.tar.gz

[ducc@oc4534203605 ~]$ umask 022
[ducc@oc4534203605 ~]$ tar -zxf uima-ducc-2.0.0-bin.tar.gz -C ~ducc

[ducc@oc4534203605 ~]$ cd apache-uima-ducc-2.0.0/admin/
[[ducc@oc4534203605 admin]$ ./ducc_post_install 
Using DUCC HOME: /home/ducc/apache-uima-ducc-2.0.0 

Existing /home/ducc/apache-uima-ducc-2.0.0/resources/site.ducc.properties moved to /home/ducc/apache-uima-ducc-2.0.0/resources/site.ducc.properties.bak
Existing /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.properties moved to /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.properties.bak
Python version:
2.6.6 (r266:84292, Nov 21 2013, 10:50:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]
Enter hostname of DUCC head[oc4534203605.ibm.com]
Ducc head is configured as oc4534203605.ibm.com 

Existing /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.nodes moved to /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.nodes.bak
Initial /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.nodes created.
Existing /home/ducc/apache-uima-ducc-2.0.0/resources/jobdriver.nodes moved to /home/ducc/apache-uima-ducc-2.0.0/resources/jobdriver.nodes.bak
Initial nodepool /home/ducc/apache-uima-ducc-2.0.0/resources/jobdriver.nodes created.

ActiveMQ is automanaged on node  oc4534203605.ibm.com
Enter full path to the Java executable: [/usr/bin/java]
Initialized property "ducc.jvm" to /usr/bin/java
JAVA: java version "1.7.0"
JAVA: Java(TM) SE Runtime Environment (build pxa6470sr9-20150417_01(SR9))
JAVA: IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 Compressed References 20150406_242981 (JIT enabled, AOT enabled)
JAVA: J9VM - R26_Java726_SR9_20150406_1443_B242981
JAVA: JIT  - tr.r11_20150401_88894
JAVA: GC   - R26_Java726_SR9_20150406_1443_B242981_CMPRSS
JAVA: J9CL - 20150406_242981)
JAVA: JCL - 20150414_02 based on Oracle 7u79-b14
Java version: "1.7.0"
Java is verified.
Enter password for webserver keystore:whyaducc
keystore created as whyaducc

Web server keystore generated from ducc.properties

DUCC book installed into webserver root

Merging /home/ducc/apache-uima-ducc-2.0.0/resources/default.ducc.properties with /home/ducc/apache-uima-ducc-2.0.0/resources/site.ducc.properties into /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.properties
Using /usr/bin/java
/usr/bin/java -cp ../lib/uima-ducc/uima-ducc-common-2.0.0.jar org.apache.uima.ducc.common.utils.OsArch
Os Architecture: amd64
cc -g    ducc_ling.c   -o ducc_ling
050 ducc_ling Version 2.0.0 compiled Jul 21 2015 at 08:44:47
ducc_ling is installed for architecture amd64. See the installation guide for multi-user setup.
Initial DUCC setup complete.

[ducc@oc4534203605 admin]$ ./check_ducc
Set ducc_ling version from oc4534203605.ibm.com : /home/ducc/apache-uima-ducc-2.0.0/admin/amd64/ducc_ling -v >/home/ducc/apache-uima-ducc-2.0.0/state/duccling.version
050 ducc_ling Version 2.0.0 compiled Jul 21 2015 at 08:44:47
JVM: /usr/bin/java
ENV: Java is configured as: /usr/bin/java
ENV java full version JRE 1.7.0 IBM Linux build pxa6470sr9-20150417_01(SR9)
ENV: Threading enabled: True
MEM: memory is 15 gB
ENV: system is Linux
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
ActiveMQ broker is not running on tcp://oc4534203605.ibm.com:61617
Checking 1 nodes
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com ps -eo user:14,pid,comm,args
Waiting for Completion
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /home/ducc/apache-uima-ducc-2.0.0/admin/check_ducc -x 1437566853
 
Checking oc4534203605.ibm.com ...
    no processes found.
    WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
    ducc_ling module /home/ducc/apache-uima-ducc-2.0.0/admin/amd64/ducc_ling : setuid bit is not set. Processes will run as user ducc
    Node health checks return.
All threads returned
[ducc@oc4534203605 admin]$ 

[degenaro@oc4534203605 ~]$ sudo su root
[sudo] password for degenaro: 
[root@oc4534203605 degenaro]# cd ~ducc/apache-uima-ducc-2.0.0/admin/
[root@oc4534203605 admin]# ls
amd64           ducc_boot.py       ducc_props_manager  ducc_reap_rogues.sh  ducc_util.pyc   local_hooks.pyc  rm_qload        sign_jconsole_jar  vary_off
build_duccling  ducc_post_install  ducc.py             ducc_statedump       jconsole.xml    pam_test         rm_qoccupancy   start_ducc         vary_on
check_ducc      ducc_props_diff    ducc.pyc            ducc_util.py         local_hooks.py  read_nodes.py    rm_reconfigure  stop_ducc
[root@oc4534203605 admin]# chown ducc.ducc amd64/
[root@oc4534203605 admin]# chmod 700 amd64/
[root@oc4534203605 admin]# chown root.ducc amd64/ducc_ling 
[root@oc4534203605 admin]# chmod 4750 amd64/ducc_ling 
[root@oc4534203605 admin]# exit
exit
[degenaro@oc4534203605 ~]$ 

[ducc@oc4534203605 admin]$ ./start_ducc
Merging /home/ducc/apache-uima-ducc-2.0.0/resources/default.ducc.properties with /home/ducc/apache-uima-ducc-2.0.0/resources/site.ducc.properties into /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.properties
Set ducc_ling version from oc4534203605.ibm.com : /home/ducc/apache-uima-ducc-2.0.0/admin/amd64/ducc_ling -v >/home/ducc/apache-uima-ducc-2.0.0/state/duccling.version
JVM: /usr/bin/java
ENV: Java is configured as: /usr/bin/java
ENV java full version JRE 1.7.0 IBM Linux build pxa6470sr9-20150417_01(SR9)
ENV: Threading enabled: True
MEM: memory is 15 gB
ENV: system is Linux
allnodes /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.nodes
Class definition file is ducc.classes
OK: Class and node definitions validated.
OK: Class configuration checked
WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
Starting broker on oc4534203605.ibm.com
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c broker '
----------- INFO: Using default configuration
[] INFO: Using default configuration
----------- (you can configure options in one of these file: /etc/default/activemq /home/ducc/.activemqrc)
[] (you can configure options in one of these file: /etc/default/activemq /home/ducc/.activemqrc)
----------- 
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
Waiting for broker ..... 0
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
Waiting for broker ..... 1
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com /bin/netstat -an
ActiveMQ broker is found on configured host and port: oc4534203605.ibm.com:61617
Starting 1 agents
********** Starting agents from file /home/ducc/apache-uima-ducc-2.0.0/resources/ducc.nodes
Starting warmssh -o BatchMode=yes -o ConnectTimeout=10 
oc4534203605.ibm.com ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -cagent -b -d 1437567159.08 --nodup '
ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.comssh -o BatchMode=yes -o ConnectTimeout=10 ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c sm -b --or_parms=warm -d 1437567159.08 --nodup '
 ssh -o BatchMode=yes -o ConnectTimeout=10 ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c rm -b --or_parms=warm -d 1437567159.08 --nodup 'Waiting for Completion
oc4534203605.ibm.com
 ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com  ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c ws -b --or_parms=warm -d 1437567159.09 --nodup '' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c pm -b --or_parms=warm -d 1437567159.08 --nodup '

 oc4534203605.ibm.com ' /home/ducc/apache-uima-ducc-2.0.0/admin/ducc.py -c or -b --or_parms=warm -d 1437567159.09 --nodup '
oc4534203605.ibm.com Starting pm
     WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
     PID 12028
oc4534203605.ibm.com Starting rm
     WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
     PID 12029
oc4534203605.ibm.com Starting ws
     WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
     PID 12030
oc4534203605.ibm.com Starting sm
     WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
     PID 12034
oc4534203605.ibm.com Starting or
     WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
     PID 12035
oc4534203605.ibm.com 
    WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
    ducc_ling module /home/ducc/apache-uima-ducc-2.0.0/admin/amd64/ducc_ling : setuid bit is not set. Processes will run as user ducc
    ssh -o BatchMode=yes -o ConnectTimeout=10 oc4534203605.ibm.com ps -eo user:14,pid,comm,args
    WARN: Soft limit RLIMIT_NOFILES is too small at 4096 (< 8192 ).  DUCC may be unable to open sufficient files or sockets.
    DUCC Agent started PID 12132
All threads returned
[ducc@oc4534203605 admin]$ 

[degenaro@oc4534203605 ~]$ 
[degenaro@oc4534203605 bin]$ export PATH=$PATH:$PWD
[degenaro@oc4534203605 bin]$ cd ~
[degenaro@oc4534203605 ~]$ mkdir ducc-test
[degenaro@oc4534203605 ~]$ cd ducc-test
[degenaro@oc4534203605 ducc-test]$ cp -pr ~ducc/apache-uima-ducc-2.0.0/examples/ .
[degenaro@oc4534203605 ducc-test]$ cd examples/simple/
[degenaro@oc4534203605 simple]$ ducc_submit -f 1.job
Job 1 submitted
[degenaro@oc4534203605 simple]$ 

[degenaro]$ sudo su root
[root]# chkconfig sshd on
[root]# /sbin/service sshd restart
[root]# exit
[degenaro]$

[degenaro]$ sudo adduser ducc

[degenaro]$ sudo su root
[root]# cd /home
[root]# chmod 755 degenaro
[root]# chmod 755 ducc
[root]# exit
[degenaro]$

 
[degenaro]$ sudo su ducc 
[ducc]$ cd
[ducc]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ducc/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ducc/.ssh/id_rsa.
Your public key has been saved in /home/ducc/.ssh/id_rsa.pub.
The key fingerprint is:
4f:4a:81:60:c9:a5:d8:88:80:f3:6c:a1:ef:31:8b:8e ducc@hal9000.apache.com
The key's randomart image is:
+--[ RSA 2048]----+
|o  .oo.          |
|+..=oo .         |
|.=o.o . .        |
|. +      .       |
| o      S .      |
|  +    . +       |
| o +    . .      |
|o o              |
|E.               |
+-----------------+
[ducc]$ cd .ssh
[ducc]$ cat id_rsa.pub >> authorized_keys2
[ducc]$ chmod 600 *
[ducc]$ exit
[degenaro]$

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions

export JAVA_HOME=~/apps/java/java-home
export PATH=$PATH:/usr/local/tex

[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk ~/svn/ducc/trunk

[degenaro]$ cd ~/svn/ducc/trunk
[degenaro]$ mvn clean install -Pbuild-duccdocs

[degenaro]$ cd target
[degenaro]$ sudo su ducc
[ducc]$ umask 022
[ducc]$ tar -zxf uima-ducc-1.0.0-SNAPSHOT-bin.tar.gz -C ~ducc
[ducc]$ cd ~
[ducc]$ ln -s apache-uima-ducc-1.0.0-SNAPSHOT ducc_install 
[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./ducc_post_install

[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./check_ducc -s
[ducc]$ exit
[degenaro]$

[degenaro]$ sudo su root
[root]$ mkdir /local
[root]$ mkdir /local/ducc
[root]$ mkdir /local/ducc/bin
[root]$ chown ducc.ducc /local/ducc
[root]$ chown ducc.ducc /local/ducc/bin
[root]$ chmod 700 /local/ducc
[root]$ chmod 700 /local/ducc/bin
[root]$ cp ~ducc/ducc_install/duccling/src/ducc_ling /local/ducc/bin
[root]$ chown root.ducc /local/ducc/bin/ducc_ling
[root]$ chmod 4750 /local/ducc/bin/ducc_ling
[root]$ exit
[degenaro]$

[degenaro]$ sudo su ducc
[ducc]$ cd ~ducc/ducc_install/resources
1. edit ducc.properties
2. locate: ducc.ws.login.enabled = false
3. modify: ducc.ws.login.enabled = true
4. locate: ducc.agent.launcher.ducc_spawn_path=${DUCC_HOME}/admin/ducc_ling
5. modify: ducc.agent.launcher.ducc_spawn_path=/local/ducc/bin/ducc_ling
6. save

[ducc]$ cd ~ducc/ducc_install/admin
[ducc]$ ./start_ducc -s
[ducc]$ exit
[degenaro]$

[degenaro]$ cd ~ducc/ducc_install/bin
[degenaro]$ export PATH=$PATH:$PWD
[degenaro]$ cd ~
[degenaro]$ mkdir ducc
[degenaro]$ cd ducc
[degenaro]$ cp -pR ~ducc/ducc_install/examples .
[degenaro]$ cd examples/simple
[degenaro]$ ducc_submit -f 1.job
Job 1 submitted
[degenaro]$

[degenaro]$ scp uima-ducc-2.1.0-bin.tar.gz degenaro@uima-ducc-vm2.apache.org:~degenaro

[degenaro@oc0883280806 Downloads]$ ssh degenaro@uima-ducc-vm2.apache.org
...
degenaro@uima-ducc-vm2:~$ 

degenaro@uima-ducc-vm2:~$ sudo su - duccy
otp-md5 451 ui3416
Password:
duccy@uima-ducc-vm2:~$ 

duccy@uima-ducc-vm2:~$ crontab -r

duccy@uima-ducc-vm2:~$ killall -9 python

duccy@uima-ducc-vm2:~$ killall -9 java

duccy@uima-ducc-vm2:~$ killall -9 service

duccy@uima-ducc-vm2:~$ cd install
duccy@uima-ducc-vm2:~/install$ rm -fr *
duccy@uima-ducc-vm2:~/install$ cp -p ~degenaro/uima-ducc-2.1.0-bin.tar.gz .
duccy@uima-ducc-vm2:~/install$ umask 022
duccy@uima-ducc-vm2:~/install$ tar -zxf *
duccy@uima-ducc-vm2:~/install$ cd
duccy@uima-ducc-vm2:~$ 

duccy@uima-ducc-vm2:~$ cd ~/ducc/logs
duccy@uima-ducc-vm2:~/ducc/logs$ rm -fr *
duccy@uima-ducc-vm2:~$ cd ~/spool
duccy@uima-ducc-vm2:~/spool$ rm -fr * 
duccy@uima-ducc-vm2:~$ cd ~/dump
duccy@uima-ducc-vm2:~/dump$ rm -fr * 
duccy@uima-ducc-vm2:~/dump$ cd
duccy@uima-ducc-vm2:~$ 

duccy@uima-ducc-vm2:~$ cd ~/ducc_runtime/admin
duccy@uima-ducc-vm2:~/ducc_runtime/admin$ ./ducc_post_install
Using DUCC HOME: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT 

Python version:
2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2]
Enter hostname of DUCC head[uima-ducc-vm]localhost
Ducc head is configured as localhost

Initial /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.nodes created.
Enter full path to the Java executable: [/opt/java/bin/java]/opt/oracle/jdk1.8.0_102/bin/java
Initialized property "ducc.jvm" to /opt/oracle/jdk1.8.0_102/bin/java
JAVA: java version "1.8.0_102"
JAVA: Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
JAVA: Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
java_home /opt/oracle/jdk1.8.0_102
ActiveMQ is automanaged on node  uima-ducc-vm
broker configuration edit
ducc.broker.admin.username=admin

ducc.broker.admin.password=xxxxxxxxxxxxxxxx

Database host is configured at listen_address: DUCCHOST
Must reconfigure listen_address to uima-ducc-vm
Starting the database.  This might take a few moments if it is the first time.
Database is started.  Waiting for initialization
:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/cassandra/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/guava/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-log4j/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*
/usr/local/java-home/bin/java -DDUCC_HOME=/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT org.apache.uima.ducc.database.DbCreate uima-ducc-vm ducc xFpMnJ3aQ
log4j:WARN No appenders could be found for logger (com.datastax.driver.core.SystemProperties).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
connect Created database super user ducc
connect Changed default database super user's password and revoked its superuser authority.
connect From this point, this DB can only be accessed in super user mode by user 'ducc'
connect Connected to cluster: DUCC
connect Datatacenter: datacenter1 Host: uima-ducc-vm/140.211.11.134 Rack: rack1
createSchema Created database user 'guest' with SELECT priveleges on DUCC tables.
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmnodes (name varchar,memory int,nodepool varchar,shares_left int,responsive boolean,online boolean,reservable boolean,ip varchar,quantum int,classes varchar,share_order int,blacklisted boolean,heartbeats int,assignments int,np_assignments int, PRIMARY KEY(name))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_responsive_idx ON rmnodes(responsive)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_online_idx ON rmnodes(online)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_reservable_idx ON rmnodes(reservable)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmshares (node varchar,ducc_dbid bigint,job_id bigint,uuid uuid,share_order int,quantum int,init_time bigint,evicted boolean,purged boolean,fixed boolean,blacklisted boolean,state varchar,pid bigint,jobtype varchar,investment bigint, PRIMARY KEY(node,ducc_dbid,job_id))
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmload (class varchar,job_id bigint,user varchar,memory int,state varchar,demand int,occupancy int,jobtype varchar, PRIMARY KEY(job_id))
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS ducc.smreg (numeric_id varchar,uuid varchar,is_archived boolean,description varchar,administrators varchar,scheduling_class varchar,log_directory varchar,working_directory varchar,jvm varchar,process_jvm_args varchar,classpath varchar,environment varchar,process_memory_size varchar,process_descriptor_DD varchar,process_debug varchar,process_executable varchar,process_executable_args varchar,process_initialization_time_max varchar,service_dependency varchar,service_request_endpoint varchar,service_linger varchar,service_ping_arguments varchar,service_ping_class varchar,service_ping_classpath varchar,service_ping_jvm_args varchar,service_ping_dolog varchar,service_ping_timeout varchar,instance_failures_window varchar,instance_failures_limit varchar,instance_init_failures_limit varchar, PRIMARY KEY(numeric_id))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS ON ducc.smreg(is_archived)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS ducc.smmeta (numeric_id varchar,is_archived boolean,uuid varchar,reference varchar,enabled varchar,autostart varchar,instances varchar,endpoint varchar,disable_reason varchar,implementors varchar,ping_active varchar,ping_only varchar,service_alive varchar,service_class varchar,service_dependency varchar,service_healthy varchar,service_state varchar,last_use varchar,last_use_readable varchar,service_statistics varchar,service_type varchar,submit_error varchar,user varchar,references varchar,stopped varchar,last_ping varchar,last_ping_readable varchar,last_runnable varchar,last_runnable_readable varchar,work_instances varchar,registration_date varchar,registration_date_millis varchar, PRIMARY KEY(numeric_id))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS ON ducc.smmeta(is_archived)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS job_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS job_history_history_idx ON job_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS res_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS res_history_history_idx ON res_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS svc_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS svc_history_history_idx ON svc_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS orckpt (id int,work blob,p2jmap blob, PRIMARY KEY(id))
Database is initialized.
Stopping the database.
kill -TERM `cat /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/state/cassandra.pid`
Configuring DUCC to use the database.
Java version: "1.7.0_79"
Java is verified.
keystore =  /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/etc/keystore

Web server keystore generated from ducc.properties

DUCC book installed into webserver root

Merging /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/default.ducc.properties with /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties into /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
Using /usr/local/java-home/bin/java
/usr/local/java-home/bin/java -cp ../lib/uima-ducc/uima-ducc-common-2.1.0-SNAPSHOT.jar org.apache.uima.ducc.common.utils.OsArch
Os Architecture: amd64
cc -g    ducc_ling.c   -o ducc_ling
050 ducc_ling Version 2.1.0 compiled Jul  6 2016 at 18:12:55
ducc_ling is installed for architecture amd64. See the installation guide for additional modifications needed to run user processes with their OS user credentials.
Initial DUCC setup complete.
duccy@uima-ducc-vm:~/ducc_runtime/admin$ 

duccy@uima-ducc-vm2:~/ducc_runtime/admin$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./installer
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/conf/activemq-ducc.xml
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.administrators
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/ducc.nodes
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ 

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ cd ~/ducc_runtime/examples/systemtest
duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$ ./start_sim --nothreading -n ducc.nodes -c all
Merging /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/default.ducc.properties with /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties into /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
Running as 22281
Set ducc_ling version from uima-ducc-vm : /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/admin/amd64/ducc_ling -v >/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/state/duccling.version
-------- start broker
broker host localhost
Waiting for broker 0
Waiting for broker 1
Waiting for broker 2
Waiting for broker 3
-------- start database
Starting database
Database host is configured at listen_address: uima-ducc-vm
Must reconfigure listen_address to localhost
waiting for database to start
Connected to cluster: DUCC
Tables found: smmeta smreg job_history svc_history rmnodes rmshares rmload res_history orckpt 
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
OK
-------- specials_only False
-------- start agents
Starting from nodes in ducc.nodes
Starting agent on uima-ducc-demo instance 1 as pseudo-node uima-ducc-demo-1 IP 192.168.4.1 memory 15
  ... Started,  PID 22620
Starting agent on uima-ducc-demo instance 2 as pseudo-node uima-ducc-demo-2 IP 192.168.4.2 memory 15
  ... Started,  PID 22675
Starting agent on uima-ducc-demo instance 3 as pseudo-node uima-ducc-demo-3 IP 192.168.4.3 memory 15
  ... Started,  PID 22728
Starting agent on uima-ducc-demo instance 4 as pseudo-node uima-ducc-demo-4 IP 192.168.4.4 memory 15
  ... Started,  PID 22779
Starting agent on uima-ducc-demo instance 5 as pseudo-node uima-ducc-demo-5 IP 192.168.4.5 memory 30
  ... Started,  PID 22836
Starting agent on uima-ducc-demo instance 6 as pseudo-node uima-ducc-demo-6 IP 192.168.4.6 memory 30
  ... Started,  PID 22895
Starting agent on uima-ducc-demo instance 7 as pseudo-node uima-ducc-demo-7 IP 192.168.4.7 memory 30
  ... Started,  PID 22976
Starting agent on uima-ducc-demo instance 8 as pseudo-node uima-ducc-demo-8 IP 192.168.4.8 memory 45
  ... Started,  PID 23043
Starting agent on uima-ducc-demo instance 9 as pseudo-node uima-ducc-demo-9 IP 192.168.4.9 memory 45
  ... Started,  PID 23128
Starting agent on uima-ducc-demo instance 10 as pseudo-node uima-ducc-demo-10 IP 192.168.4.10 memory 60
  ... Started,  PID 23212
Start ws on uima-ducc-vm
    PID 23234
Start sm on uima-ducc-vm
    PID 23258
Start rm on uima-ducc-vm
    PID 23286
Start or on uima-ducc-vm
    PID 23322
Start pm on uima-ducc-vm
    PID 23343
All Work completed
duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$

duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver/
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./prep
yak
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 0 --description "Test Service 0 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 0 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_0.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_0:tcp://localhost:61617 ID[0]
registered service: 0
chinchilla
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 1 --description "Test Service 1 Type auto-start"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 1 Type auto-start' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_1.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_1:tcp://localhost:61617 ID[1]
registered service: 1
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --autostart true --modify 1
Service Modify succeeded - Modify accepted: - UIMA-AS:FixedSleepAE_1:tcp://localhost:61617 ID[1]
autostart service: 1
roadrunner
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 2 --description "Test Service 2 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 2 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_2.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_2:tcp://localhost:61617 ID[2]
registered service: 2
guppy
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 3 --description "Test Service 3 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 3 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_3.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_3:tcp://localhost:61617 ID[3]
registered service: 3
kiwi
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 4 --description "Test Service 4 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 4 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_4.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_4:tcp://localhost:61617 ID[4]
registered service: 4
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ 

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ cd ~/ducc_runtime/examples/uima-ducc-vm/custom
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/custom$ make
cc -g    service.c   -o service
javac -classpath ~/ducc_runtime/lib/uima-ducc-cli.jar CustomPing.java
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/custom$ ./customsvc
Service setup using
   project_home  : /home/duccy/ducc_runtime/examples/uima-ducc-vm
   DUCC_HOME     : /home/duccy/ducc_runtime
   autostart     : True
   CLASSPATH:
     /home/duccy/ducc_runtime/examples/uima-ducc-vm/custom
Register new service
/home/duccy/ducc_runtime/bin/ducc_services --register --process_executable 'service' --service_request_endpoint 'CUSTOM:localhost:7175' --service_ping_class 'CustomPing' --description 'Custom Pinger' --autostart 'True' --service_ping_timeout '10000' --service_ping_classpath '/home/duccy/ducc_runtime/examples/uima-ducc-vm/custom' --service_ping_dolog 'false' --process_executable_args '7175' 
Service Register succeeded - Registered. - CUSTOM:localhost:7175 ID[5]

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/custom$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./driver_start
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ nohup: appending output to ‘nohup.out’
...

duccy@uima-ducc-vm2:~$ cd ~/ducc_runtime/examples/uima-ducc-vm/cron
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/cron$ crontab crontab.txt

[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/uima-ducc/trunk
...
[degenaro]$ mvn clean install -Pbuild-duccdocs