...
- java >= 8
- curl (or postman or something similar)
- Unzip tika-pipes-tutorial-20211202.zip
- In
tika-pipes-tutorial-20220124/app-bin/
:- https://dlcdn.apache.org/tika/2.2.1/tika-app-2.2.1.jar
- https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-fs/2.2.1/tika-emitter-fs-2.2.1.jar
- https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-solr/2.2.1/tika-emitter-solr-2.2.1.jar OR https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-opensearch/2.2.1/tika-emitter-opensearch-2.2.1.jar
- https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/437/org.apache.tika$tika-core/artifact/org.apache.tika/tika-core/2.2.2-20220124.115541-55/tika-core-2.2.2-20220124.115541-55-test-jar-with-dependencies.jar
- Optional: In
tika-pipes-tutorial-20220124/server-bin/
:- tika-server-standard jar: https://dlcdn.apache.org/tika/2.2.1/tika-server-standard-2.2.1.jar
- https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-fs/2.2.1/tika-emitter-fs-2.2.1.jar
- https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-solr/2.2.1/tika-emitter-solr-2.2.1.jar OR https://repo1.maven.org/maven2/org/apache/tika/tika-emitter-opensearch/2.2.1/tika-emitter-opensearch-2.2.1.jar
- https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/437/org.apache.tika$tika-core/artifact/org.apache.tika/tika-core/2.2.2-20220124.115541-55/tika-core-2.2.2-20220124.115541-55-test-jar-with-dependencies.jar
- Installation of Apache Solr (~8.9.x) and/or OpenSearch (~1.x) and/or Elasticsearch (7.x)
A) Fileshare to Fileshare warm up
- Run the async processor via tika-app
Configure the
basePath
element inFileSystemPipesIterator
andFileSystemPipesIterator
inconfigs/tika-config-app-fs-to-fs.xml
Code Block language xml title FileSystemPipesIterator collapse true <fetcher class="org.apache.tika.pipes.fetcher.fs.FileSystemFetcher"> <params> <name>fsf</name> <basePath>/Users/allison/Desktop/tika-pipes-tutorial-20220124/docs</basePath> </params> </fetcher> <pipesIterator class="org.apache.tika.pipes.pipesiterator.fs.FileSystemPipesIterator"> <params> <fetcherName>fsf</fetcherName> <emitterName>fse</emitterName> <basePath>/Users/allison/Desktop/tika-pipes-tutorial-20220124/docs</basePath> </params> </pipesIterator>
Configure the
basePath
element inFileSystemPipesEmitter
inconfigs/tika-config-app-fs-to-fs.xml
Code Block language xml title FileSystemPipesIterator collapse true <emitters> <emitter class="org.apache.tika.pipes.emitter.fs.FileSystemEmitter"> <params> <name>fse</name> <basePath>/Users/allison/Desktop/tika-pipes-tutorial-20220124/extracts</basePath> </params> </emitter> </emitters>
java -cp "app-bin/*" org.apache.tika.cli.TikaCLI -a --config=configs/tika-config-app-fs-to-fs.xml
B) OpenSearch/Elasticsearch Parent-Child Example (fileshare to OpenSearch/ElasticSearch)
- Start opensearch via Docker:
- docker pull opensearchproject/opensearch:1.2.0
- docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.2.0
- Curl schema to opensearch:
curl -k -T configs/opensearch/opensearch-parent-child-mappings.json -u admin:admin -H "Content-Type:application/json" https://localhost:9200/tika-test-parent-child
Configure the
basePath
element inFileSystemPipesIterator
a nd andFileSystemFetcher in
configs/opensearch/tika-config-fs-to-opensearch-parent-child.xml
java -cp "app-bin/*" org.apache.tika.cli.TikaCLI -a --config=configs/opensearch/tika-config-fs-to-opensearch-parent-child.xml
C) OpenSearch/Elasticsearch Individual Files Example (fileshare to OpenSearch/ElasticSearch)
- Start opensearch via Docker:
- docker pull opensearchproject/opensearch:1.2.0
- docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.2.0
- Curl schema to opensearch:
curl -k -T configs/opensearch/opensearch-indiv-files-mappings.json -u admin:admin -H "Content-Type:application/json" https://localhost:9200/tika-test-indiv-files
Configure the
basePath
element inFileSystemPipesIterator
andFileSystemFetcher in
configs/opensearch/tika-config-fs-to-opensearch-indiv-files.xml
java -cp "app-bin/*" org.apache.tika.cli.TikaCLI -a --config=configs/opensearch/tika-config-fs-to-opensearch-indiv-files.xml
D) OpenSearch/Elasticsearch Legacy Example (fileshare to OpenSearch/ElasticSearch)
- Start opensearch via Docker:
- docker pull opensearchproject/opensearch:1.2.0
- docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.2.0
- Curl schema to opensearch:
curl -k -T configs/opensearch/opensearch-indiv-legacy.json -u admin:admin -H "Content-Type:application/json" https://localhost:9200/tika-test-indiv-https://localhost:9200/tika-test-legacy
Configure the
basePath
element inFileSystemPipesIterator
andFileSystemFetcher in
configs/opensearch/tika-config-fs-to-opensearch-legacy.xml
java -cp "app-bin/*" org.apache.tika.cli.TikaCLI -a --config=configs/opensearch/tika-config-fs-to-opensearch-legacy.xml