Page History

...

Anchor

	t0
	t0

Tutotial: Quick Start (from gz)

...

Code Block
[degenaro]$ scp uima-ducc-2.01.0-bin.tar.gz degenaro@uima-ducc-vmvm2.apache.org:~degenaro

Login to installation host

...

Code Block
[degenaro@oc0883280806 Downloads]$ ssh degenaro@uima-ducc-vmvm2.apache.org ... degenaro@uima-ducc-vm:~$ -ducc-vm2:~$

Switch to user duccy (note: use https://reference.apache.org/committer/otp-md5 for password challenge)

Code Block
degenaro@uima-ducc-vm2:~$ sudo su - duccy otp-md5 451 ui3416 Password: duccy@uima-ducc-vm2:~$

Remove crontab

Code Block
duccy@uima-ducc-vm2:~$ crontab -r

Stop demo (if running)

Code Block
degenaro@uimaduccy@uima-ducc-vmvm2:~$ killall -9 python

Stop ducc (if running)

Code Block
degenaro@uimaduccy@uima-ducc-vmvm2:~$ killall -9 java

Stop custom service and/or sleepers (if running)

Code Block
degenaro@uimaduccy@uima-ducc-vmvm2:~$ killall -9 service

Install new code

Code Block

degenaro@uimaduccy@uima-ducc-vmvm2:~$ cd ~/ducc/versions
degenaro@uimainstall
duccy@uima-ducc-vmvm2:~/ducc/versions$install$ rm -fr trash/*
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/versions$install$ mv u* trash
degenaro@uima-ducc-vm:~/ducc/versions$ mv ~cp -p ~degenaro/uima-ducc-2.01.0-bin.tar.gz .
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/versions$install$ umask 022
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/versions$install$ tar -zxf *gz
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/versions$install$ lscd
apache-uimaduccy@uima-ducc-2.0.0  trash  uima-ducc-2.0.0-bin.tar.gzvm2:~$

...

remove old logs and dumps

Code Block

degenaro@uimaduccy@uima-ducc-vmvm2:~$ cd ~/ducc/logs
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/ducc$logs$ rm release
degenaro@uima-fr *
duccy@uima-ducc-vmvm2:~$ cd ~/ducc$ rm 1.1.0
degenaro@uimaspool
duccy@uima-ducc-vmvm2:~/ducc$spool$ lnrm -s versions/apache-uimafr * 
duccy@uima-ducc-2.0.0 2.0.0
degenaro@uimavm2:~$ cd ~/dump
duccy@uima-ducc-vmvm2:~/ducc$dump$ lnrm -s 2.0.0 release
degenaro@uimafr * 
duccy@uima-ducc-vmvm2:~/ducc$ ls -atl
drwxr-xr-x  3 degenaro degenaro 4096 Oct  7 13:06 .
lrwxrwxrwx  1 degenaro degenaro    5 Oct  7 13:00 release -> 2.0.0
lrwxrwxrwx  1 degenaro degenaro   31 Oct  7 12:59 2.0.0 -> versionsdump$ cd
duccy@uima-ducc-vm2:~$

Run ducc_post_install

Code Block

duccy@uima-ducc-vm2:~$ cd ~/ducc_runtime/admin
duccy@uima-ducc-vm2:~/ducc_runtime/admin$ ./ducc_post_install
Using DUCC HOME: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0
drwxr-xr-x  4 degenaro degenaro 4096 Oct  7 12:55 versions
drwxr-xr-x 19 degenaro degenaro 4096 Oct  7 12:54 ..
lrwxrwxrwx  1 degenaro degenaro   28 Aug 27 14:44 logs -> /home/degenaro/tmp/ducc/logs
lrwxrwxrwx  1 degenaro degenaro    7 Mar 29  2014 install -> release
degenaro@uima-ducc-vm:~/ducc$ rm -fr logs/*

Run ducc_post_install

Code Block

degenaro@uima-ducc-vm:~/ducc/install/admin$ ./ducc_post_install 
Using DUCC HOME: /home/degenaro/ducc/versions/apache-uima-ducc-2.0.0-SNAPSHOT 

Python version:
2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3]
Enter hostname of DUCC head[uima-ducc-vm]
Ducc head is configured as uima-ducc-vm 

Initial /home/degenaro/ducc/versions/apache-uima-ducc-2.0.0-SNAPSHOT/resources/ducc.nodes created.
Initial nodepool /home/degenaro/ducc/versions/apache-uima-ducc-2.0.0-SNAPSHOT/resources/jobdriver.nodes created.
-SNAPSHOT 

Python version:
2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2]
Enter hostname of DUCC head[uima-ducc-vm]localhost
Ducc head is configured as localhost

Initial /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.nodes created.
Enter full path to the Java executable: [/opt/java/bin/java]/opt/oracle/jdk1.8.0_102/bin/java
Initialized property "ducc.jvm" to /opt/oracle/jdk1.8.0_102/bin/java
JAVA: java version "1.8.0_102"
JAVA: Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
JAVA: Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
java_home /opt/oracle/jdk1.8.0_102
ActiveMQ is automanaged on node  uima-ducc-vm
Enterbroker full path to the Java binary directory: [/home/degenaro/local/sun/jdk1.7.0_45/bin/java]
Initialized property "ducc.jvm" to /home/degenaro/local/sun/jdk1.7.0_45/bin/java
JAVA: java version "1.7.0_45"
JAVA: Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
JAVA: Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Java version: "1.7.0_45"
Java is verified.
Enter password for webserver keystore:<xxxxxxxx>
keystore created as xxxxxxxx

Web server keystore generated from ducc.properties

DUCC book installed into webserver root

cc -g    ducc_ling.c   -o ducc_ling
Initial ducc_ling is installed.  See the installation guide for multi-user setup.
Merging /home/degenaro/ducc/versionsconfiguration edit
ducc.broker.admin.username=admin

ducc.broker.admin.password=xxxxxxxxxxxxxxxx

Database host is configured at listen_address: DUCCHOST
Must reconfigure listen_address to uima-ducc-vm
Starting the database.  This might take a few moments if it is the first time.
Database is started.  Waiting for initialization
:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/cassandra/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0-SNAPSHOT/lib/resources/default.ducc.properties with /home/degenaroguava/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0-SNAPSHOT/resources/site.ducc.properties into /home/degenarolib/apache-log4j/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0-SNAPSHOT/resources/ducc.properties

Install local modifications

Code Block

degenaro@uima-ducc-vm:~/ducc/install/examples/lib/uima-ducc-vm/driver$ ./installer
installed: /home/degenaro/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0-SNAPSHOT/apache-uima/apache-activemq/conf/activemq-ducc.xml
installed: /home/degenaro/lib/*
/usr/local/java-home/bin/java -DDUCC_HOME=/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.01.0/resources/site-SNAPSHOT org.apache.uima.ducc.properties
installed: /home/degenaro/ducc/versions/apache-database.DbCreate uima-ducc-2.0.0/resources/ducc.administrators
installed: /home/degenaro/ducc/versions/apache-uima-ducc-2.0.0/examples/systemtest/ducc.nodes

Launch DUCC (with "sim" nodes)

Code Block

degenaro@uima-ducc-vm:~$ cd ~/ducc/install/examples/systemtest
degenaro@uima-ducc-vm:~/ducc/install/examples/systemtest$ ./start_sim --nothreading -n ducc.nodes -c all
Running as 574
Set ducc_ling version from uima-ducc-vm : /home/degenaro/ducc/versions/2.0.0/apache-uima-ducc-2.0.0/admin/ducc_ling -v >/home/degenaro/ducc/versions/2.0.0/apache-uima-ducc-2.0.0/state/duccling.version
ducc_ling OK
broker host localhost
Waiting for broker 0
Starting from nodes in ducc.nodes
Starting agent on uima-ducc-vm instance 1 as pseudo-node uima-ducc-vm-1 IP 192.168.4.1 memory 15
  ... Started,  PID 1344
Starting agent on uima-ducc-vm instance 2 as pseudo-node uima-ducc-vm-2 IP 192.168.4.2 memory 15
  ... Started,  PID 1549
Starting agent on uima-ducc-vm instance 3 as pseudo-node uima-ducc-vm-3 IP 192.168.4.3 memory 15
  ... Started,  PID 1771
Starting agent on uima-ducc-vm instance 4 as pseudo-node uima-ducc-vm-4 IP 192.168.4.4 memory 15
  ... Started,  PID 1990
Starting agent on uima-ducc-vm instance 5 as pseudo-node uima-ducc-vm-5 IP 192.168.4.5 memory 30
  ... Started,  PID 2204
Starting agent on uima-ducc-vm instance 6 as pseudo-node uima-ducc-vm-6 IP 192.168.4.6 memory 30
  ... Started,  PID 2409
Starting agent on uima-ducc-vm instance 7 as pseudo-node uima-ducc-vm-7 IP 192.168.4.7 memory 30
  ... Started,  PID 2629
Starting agent on uima-ducc-vm instance 8 as pseudo-node uima-ducc-vm-8 IP 192.168.4.8 memory 45
  ... Started,  PID 2841
Starting agent on uima-ducc-vm instance 9 as pseudo-node uima-ducc-vm-9 IP 192.168.4.9 memory 45
  ... Started,  PID 3053
Start db on uima-ducc-vm
    PID 3233
Start ws on uima-ducc-vm
    PID 3407
Start sm on uima-ducc-vm
    PID 3591
Start rm on uima-ducc-vm
    PID 3766
Start or on uima-ducc-vm
    PID 3943
Start pm on uima-ducc-vm
    PID 4123
All Work completed

Create user logs directory

Code Block

degenaro@uima-ducc-vm:~$ cd /tmp
degenaro@uima-ducc-vm:/tmp$ mkdir degenaro
degenaro@uima-ducc-vm:/tmp$ mkdir degenaro/ducc
degenaro@uima-ducc-vm:/tmp$ mkdir degenaro/ducc/logs

degenaro@uima-ducc-vm:~$ cd ~/ducc
degenaro@uima-ducc-vm:~/ducc$ ln -s /tmp/degenaro/ducc/logs/ logs
degenaro@uima-ducc-vm:~/ducc$

Wait for all DUCC daemons to be running

Visit uima-ducc-vm.apache.org:42133/system.daemons.jsp

Image Removed

Create services

vm ducc xFpMnJ3aQ
log4j:WARN No appenders could be found for logger (com.datastax.driver.core.SystemProperties).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
Waiting for database to boot ...
connect Created database super user ducc
connect Changed default database super user's password and revoked its superuser authority.
connect From this point, this DB can only be accessed in super user mode by user 'ducc'
connect Connected to cluster: DUCC
connect Datatacenter: datacenter1 Host: uima-ducc-vm/140.211.11.134 Rack: rack1
createSchema Created database user 'guest' with SELECT priveleges on DUCC tables.
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmnodes (name varchar,memory int,nodepool varchar,shares_left int,responsive boolean,online boolean,reservable boolean,ip varchar,quantum int,classes varchar,share_order int,blacklisted boolean,heartbeats int,assignments int,np_assignments int, PRIMARY KEY(name))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_responsive_idx ON rmnodes(responsive)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_online_idx ON rmnodes(online)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS rmnodes_reservable_idx ON rmnodes(reservable)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmshares (node varchar,ducc_dbid bigint,job_id bigint,uuid uuid,share_order int,quantum int,init_time bigint,evicted boolean,purged boolean,fixed boolean,blacklisted boolean,state varchar,pid bigint,jobtype varchar,investment bigint, PRIMARY KEY(node,ducc_dbid,job_id))
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS rmload (class varchar,job_id bigint,user varchar,memory int,state varchar,demand int,occupancy int,jobtype varchar, PRIMARY KEY(job_id))
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS ducc.smreg (numeric_id varchar,uuid varchar,is_archived boolean,description varchar,administrators varchar,scheduling_class varchar,log_directory varchar,working_directory varchar,jvm varchar,process_jvm_args varchar,classpath varchar,environment varchar,process_memory_size varchar,process_descriptor_DD varchar,process_debug varchar,process_executable varchar,process_executable_args varchar,process_initialization_time_max varchar,service_dependency varchar,service_request_endpoint varchar,service_linger varchar,service_ping_arguments varchar,service_ping_class varchar,service_ping_classpath varchar,service_ping_jvm_args varchar,service_ping_dolog varchar,service_ping_timeout varchar,instance_failures_window varchar,instance_failures_limit varchar,instance_init_failures_limit varchar, PRIMARY KEY(numeric_id))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS ON ducc.smreg(is_archived)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS ducc.smmeta (numeric_id varchar,is_archived boolean,uuid varchar,reference varchar,enabled varchar,autostart varchar,instances varchar,endpoint varchar,disable_reason varchar,implementors varchar,ping_active varchar,ping_only varchar,service_alive varchar,service_class varchar,service_dependency varchar,service_healthy varchar,service_state varchar,last_use varchar,last_use_readable varchar,service_statistics varchar,service_type varchar,submit_error varchar,user varchar,references varchar,stopped varchar,last_ping varchar,last_ping_readable varchar,last_runnable varchar,last_runnable_readable varchar,work_instances varchar,registration_date varchar,registration_date_millis varchar, PRIMARY KEY(numeric_id))
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS ON ducc.smmeta(is_archived)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS job_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS job_history_history_idx ON job_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS res_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS res_history_history_idx ON res_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS svc_history (type varchar,ducc_id bigint,history boolean,work blob, PRIMARY KEY(type,ducc_id))WITH CLUSTERING ORDER BY (ducc_id desc)
createSchema EXECUTE STATEMENT: CREATE INDEX IF NOT EXISTS svc_history_history_idx ON svc_history(history)
createSchema EXECUTE STATEMENT: CREATE TABLE IF NOT EXISTS orckpt (id int,work blob,p2jmap blob, PRIMARY KEY(id))
Database is initialized.
Stopping the database.
kill -TERM `cat /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/state/cassandra.pid`
Configuring DUCC to use the database.
Java version: "1.7.0_79"
Java is verified.
keystore =  /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/etc/keystore

Web server keystore generated from ducc.properties

DUCC book installed into webserver root

Merging /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/default.ducc.properties with /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties into /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
Using /usr/local/java-home/bin/java
/usr/local/java-home/bin/java -cp ../lib/uima-ducc/uima-ducc-common-2.1.0-SNAPSHOT.jar org.apache.uima.ducc.common.utils.OsArch
Os Architecture: amd64
cc -g    ducc_ling.c   -o ducc_ling
050 ducc_ling Version 2.1.0 compiled Jul  6 2016 at 18:12:55
ducc_ling is installed for architecture amd64. See the installation guide for additional modifications needed to run user processes with their OS user credentials.
Initial DUCC setup complete.
duccy@uima-ducc-vm:~/ducc_runtime/admin$

Install local modifications

Code Block

duccy@uima-ducc-vm2:~/ducc_runtime/admin$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./installer
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/conf/activemq-ducc.xml
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.administrators
installed: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/ducc.nodes
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$

Launch DUCC (with "sim" nodes)

Code Block

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ cd ~/ducc_runtime/examples/systemtest
duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$ ./start_sim --nothreading -n ducc.nodes -c all
Merging /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/default.ducc.properties with /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/site.ducc.properties into /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
Running as 22281
Set ducc_ling version from uima-ducc-vm : /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/admin/amd64/ducc_ling -v >/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/state/duccling.version
-------- start broker
broker host localhost
Waiting for broker 0
Waiting for broker 1
Waiting for broker 2
Waiting for broker 3
-------- start database
Starting database
Database host is configured at listen_address: uima-ducc-vm
Must reconfigure listen_address to localhost
waiting for database to start
Connected to cluster: DUCC
Tables found: smmeta smreg job_history svc_history rmnodes rmshares rmload res_history orckpt 
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
OK
-------- specials_only False
-------- start agents
Starting from nodes in ducc.nodes
Starting agent on uima-ducc-demo instance 1 as pseudo-node uima-ducc-demo-1 IP 192.168.4.1 memory 15
  ... Started,  PID 22620
Starting agent on uima-ducc-demo instance 2 as pseudo-node uima-ducc-demo-2 IP 192.168.4.2 memory 15
  ... Started,  PID 22675
Starting agent on uima-ducc-demo instance 3 as pseudo-node uima-ducc-demo-3 IP 192.168.4.3 memory 15
  ... Started,  PID 22728
Starting agent on uima-ducc-demo instance 4 as pseudo-node uima-ducc-demo-4 IP 192.168.4.4 memory 15
  ... Started,  PID 22779
Starting agent on uima-ducc-demo instance 5 as pseudo-node uima-ducc-demo-5 IP 192.168.4.5 memory 30
  ... Started,  PID 22836
Starting agent on uima-ducc-demo instance 6 as pseudo-node uima-ducc-demo-6 IP 192.168.4.6 memory 30
  ... Started,  PID 22895
Starting agent on uima-ducc-demo instance 7 as pseudo-node uima-ducc-demo-7 IP 192.168.4.7 memory 30
  ... Started,  PID 22976
Starting agent on uima-ducc-demo instance 8 as pseudo-node uima-ducc-demo-8 IP 192.168.4.8 memory 45
  ... Started,  PID 23043
Starting agent on uima-ducc-demo instance 9 as pseudo-node uima-ducc-demo-9 IP 192.168.4.9 memory 45
  ... Started,  PID 23128
Starting agent on uima-ducc-demo instance 10 as pseudo-node uima-ducc-demo-10 IP 192.168.4.10 memory 60
  ... Started,  PID 23212
Start ws on uima-ducc-vm
    PID 23234
Start sm on uima-ducc-vm
    PID 23258
Start rm on uima-ducc-vm
    PID 23286
Start or on uima-ducc-vm
    PID 23322
Start pm on uima-ducc-vm
    PID 23343
All Work completed
duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$

Wait for all DUCC daemons to be running

Visit uima-ducc-demo.apache.org:42133/system.daemons.jsp

Image Added

Create services

Code Block

duccy@uima-ducc-vm2:~/ducc_runtime/examples/systemtest$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver/
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./prep
yak
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 0 --description "Test Service 0 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 0 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_0.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_0:tcp://localhost:61617 ID[0]
registered service: 0
chinchilla
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 1 --description "Test Service 1 Type auto-start"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0

Code Block

degenaro@uima-ducc-vm:~$ cd ~/ducc/install/examples/uima-ducc-vm/driver
degenaro@uima-ducc-vm:~/ducc/install/examples/uima-ducc-vm/driver$ ./prep
/home/degenaro/ducc/versions/1.0.1/apache-uima-ducc-1.0.1-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 1 Type auto-start' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/lib/uima-ducc/examples/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/apache-uima/lib/uima/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/home/degenaroapache-uima/apache-activemq/lib/*:/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/uima-ducc-vm/driver' --process_DD '/home/degenaroapache-uima/apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_1.xml' --schedulingservice_ping_classarguments 'fixed'
Service Register succeeded - Registered. - UIMA-AS:FixedSleepAE_1:tcp://localhost:61617 ID[0]
registered service: 1
/home/degenarobroker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/binexamples/uima-ducc_services --register --instances 1 --description 'Test Service 2' -vm/driver' --process_jvmdescriptor_argsDD '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/home/degenaro/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/lib/uima-ducc/examples/*:simple/homeresources/degenaro/ducc/versions/1.0.1/apache-uima-ducc-1.0.1-SNAPSHOT/lib/uima/*:/home/degenaroservice/Service_FixedSleep_1.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_1:tcp://localhost:61617 ID[1]
registered service: 1
cmd: /usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service'bin/ducc_services --service_ping_arguments 'broker-jmx-port=9999'autostart true --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/home/degenaromodify 1
Service Modify succeeded - Modify accepted: - UIMA-AS:FixedSleepAE_1:tcp://localhost:61617 ID[1]
autostart service: 1
roadrunner
cmd: /usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/uima-ducc-vm/driver' --process_DD '/home/degenaro/systemtest/reg_service -i 2 --description "Test Service 2 Type on-demand"
/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_2.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered. - UIMA-AS:FixedSleepAE_2:bin/ducc_services --register --instances 1 --description 'Test Service 2 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617 ID[1]
registered service: 2
/home/degenaro' --classpath '/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 3' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/home/degenarolib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/libapache-uima/uimaapache-duccactivemq/exampleslib/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/apache-uima/apache-activemq/lib/uimaoptional/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_32.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered. - UIMA-AS:FixedSleepAE_3:tcp://localhost:61617 ID[2]
registered service: 3
/home/degenaro/ducc/versions/1.0.1/apache-uima-ducc-1.0.1-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 4' --process_jvm_args '-DdefaultBrokerURL= 'fixed'
Service Register succeeded - Registered - UIMA-AS:FixedSleepAE_2:tcp://localhost:61617' --classpath '/home/degenaro ID[2]
registered service: 2
guppy
cmd: /usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/lib/uima-ducc/examples/*:/home/degenaro/ducc/versions/1.0.1/apache-uima-ducc-1.0.1-SNAPSHOT/lib/uima/*:/home/degenaro/ducc/versions/1.0.1systemtest/reg_service -i 3 --description "Test Service 3 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service'bin/ducc_services --service_ping_arguments 'broker-jmx-port=9999'register --instances 1 --environmentdescription 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/home/degenaro/ducc/versions/1.0.1/apache-uima-ducc-1.0.1-SNAPSHOT/examples/uima-ducc-vm/driver' --process_DD '/home/degenaroTest Service 3 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_4.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered. - UIMA-AS:FixedSleepAE_4:tcp://localhost:61617 ID[3]
registered service: 4
/home/degenarolib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 5' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/home/degenaroapache-uima/lib/*:/usr/local/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/libapache-uima/uimaapache-duccactivemq/exampleslib/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/apache-uima/apache-activemq/lib/uimaoptional/*:/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service' --service_ping_arguments 'broker-jmx-port=9999' --environment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --working_directory '/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/uima-ducc-vm/driver' --process_descriptor_DD '/homeusr/degenarolocal/ducc/versions/2.1.0.1-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/simple/resources/service/Service_FixedSleep_53.xml' --scheduling_class 'fixed'
Service Register succeeded - Registered. - UIMA-AS:FixedSleepAE_53:tcp://localhost:61617 ID[43]
registered service: 5
Service Modify succeeded - Service 1 modify request accepted. - UIMA-AS:FixedSleepAE_2:tcp://localhost:61617 ID[1]
autostart service: 1

Create Custom services

Code Block

degenaro@uima-ducc-vm:~$ cd ~/ducc/install/examples/uima-ducc-vm/custom
degenaro@uima-ducc-vm:~/ducc/install/examples/uima-ducc-vm/custom$ make
cc -g    service.c   -o service
javac -classpath ~/ducc/install/lib/uima-ducc-cli.jar CustomPing.java
degenaro@uima-ducc-vm:~/ducc/install/examples/uima-ducc-vm/custom$ ./customsvc
Service setup using
   project_home  : /home/degenaro/ducc/versions3
kiwi
cmd: /usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/systemtest/reg_service -i 4 --description "Test Service 4 Type on-demand"
/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/bin/ducc_services --register --instances 1 --description 'Test Service 4 Type on-demand' --process_jvm_args '-DdefaultBrokerURL=tcp://localhost:61617' --classpath '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examplesapache-uima/uima-ducc-vm
   DUCC_HOME     : /home/degenaro/ducc/install
   autostart     : True
   CLASSPATH:
     /home/degenaro/ducc/versionsapache-activemq/lib/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examplesapache-uima/uima-ducc-vm/custom
Register new service
/home/degenaro/ducc/install/bin/ducc_services --register --process_executable 'apache-activemq/lib/optional/*:/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/simple/resources/service' --service_requestping_endpointarguments 'CUSTOM:localhost:7175broker-jmx-port=9999' --service_ping_class 'CustomPing' --description 'Custom Pingerenvironment 'AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0' --process_memory_size '15' --autostart 'True' --service_ping_timeout '10000working_directory '/usr/local/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/driver' --serviceprocess_pingdescriptor_classpathDD '/homeusr/degenarolocal/ducc/versions/2.1.0-beta/apache-uima-ducc-2.1.0.1-SNAPSHOT/examples/uima-ducc-vm/custom/simple/resources/service/Service_FixedSleep_4.xml' --servicescheduling_ping_dologclass 'false'fixed'
Service Register succeeded - Registered - UIMA-process_executable_args '7175' 
Service Register succeeded - Registered. - CUSTOM:localhost:7175 ID[5]

Start driver

AS:FixedSleepAE_4:tcp://localhost:61617 ID[4]
registered service: 4
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$

Create Custom services

Code Block

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$

Code Block

degenaro@uima-ducc-vm:~$ cd ~/ducc_runtime/install/examples/uima-ducc-vm/drivercustom
degenaro@uimaduccy@uima-ducc-vmvm2:~/ducc/install_runtime/examples/uima-ducc-vm/driver$ ./driver_start
degenaro@uima-ducc-vmcustom$ make
cc -g    service.c   -o service
javac -classpath ~/ducc_runtime/lib/uima-ducc-cli.jar CustomPing.java
duccy@uima-ducc-vm2:~/ducc/install_runtime/examples/uima-ducc-vm/driver$ nohup: appending output to `nohup.out'

degenaro@uima-ducc-vm:~/ducc/install//custom$ ./customsvc
Service setup using
   project_home  : /home/duccy/ducc_runtime/examples/uima-ducc-vm/driver$ tail -n 20 -f nohup.out 
driver.start
init:1
init:2
run.start:1
Thread-1 seconds to sleep: 124
init:3
run.start:2
Thread-2 seconds to sleep: 90
init:4
run.start:3
Thread-3 seconds to sleep: 22
init:5
run.start:4
Thread-4 seconds to sleep: 243
run.start:5
Thread-5 seconds to sleep: 225
   DUCC_HOME     : /home/duccy/ducc_runtime
   autostart     : True
   CLASSPATH:
     /home/duccy/ducc_runtime/examples/uima-ducc-vm/custom
Register new service
/home/duccy/ducc_runtime/bin/ducc_services --register --process_executable 'service' --service_request_endpoint 'CUSTOM:localhost:7175' --service_ping_class 'CustomPing' --description 'Custom Pinger' --autostart 'True' --service_ping_timeout '10000' --service_ping_classpath '/home/duccy/ducc_runtime/examples/uima-ducc-vm/custom' --service_ping_dolog 'false' --process_executable_args '7175' 
Service Register succeeded - Registered. - CUSTOM:localhost:7175 ID[5]

Start driver

Code Block

duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/custom$ cd ~/ducc_runtime/examples/uima-ducc-vm/driver
duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/driver$ ./driver_start
duccy@uima-ducc-vm2:~/ducc_runtime
dodo
['/home/degenaro/ducc/versions/apache-uima-ducc-1.0.1-SNAPSHOT/bin/ducc_submit', '--wait_for_completion', '--scheduling_class', 'normal', '--process_memory_size', '30', '--process_thread_count', '6', '--log_directory', '/tmp/ducc/driver/dodo/ducc/logs', '--working_directory', '/tmp/ducc/driver/dodo/ducc/work/3', '-f', '/home/degenaro/ducc/versions/apache-uima-ducc-1.0.1-SNAPSHOT/examples/uima-ducc-vm/jobs/2.job']driver$ nohup: appending output to ‘nohup.out’
...

Check for Jobs to be running

Visit uima-ducc-vm.apache.org:42133/jobs.jsp

...

Image Added

Install crontab (for auto-restart and auto-cleanup)

Code Block
duccy@uima-ducc-vm2:~$ cd ~/ducc_runtime/examples/uima-ducc-vm/cron duccy@uima-ducc-vm2:~/ducc_runtime/examples/uima-ducc-vm/cron$ crontab crontab.txt

End.

...

Anchor

	t3
	t3

Tutorial: Check out 2.1.0 & build

Code Block
[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/uima-ducc/trunk ... [degenaro]$ mvn clean install -Pbuild-duccdocs

End.

...

cd

Child pages

Versions Compared

Old Version 170

New Version Current

Key

Tutotial: Quick Start (from gz)

Login to installation host

Switch to user duccy (note: use https://reference.apache.org/committer/otp-md5 for password challenge)

Remove crontab

Stop demo (if running)

Stop ducc (if running)

Stop custom service and/or sleepers (if running)

Install new code

...

remove old logs and dumps

Run ducc_post_install

Run ducc_post_install

Install local modifications

Launch DUCC (with "sim" nodes)

Create user logs directory

Wait for all DUCC daemons to be running

Visit uima-ducc-vm.apache.org:42133/system.daemons.jsp

Create services

Install local modifications

Launch DUCC (with "sim" nodes)

Visit uima-ducc-demo.apache.org:42133/system.daemons.jsp

Create services

Create Custom services

Start driver

Create Custom services

Start driver

Check for Jobs to be running

Visit uima-ducc-vm.apache.org:42133/jobs.jsp

Image Added

Install crontab (for auto-restart and auto-cleanup)

Tutorial: Check out 2.1.0 & build