Page History

Instructions to train and run a simple parts-of-speech (PoS) tagger program. Instructions are for Unix, but adaptable for Windows.

...

1. Download and install Java.
2. Download and install Maven.
3. Download OpenNLP.
4. Download a PoS Treebank training set into $HOME/archives/pos.
   • ERG from DELPH-IN - $0
   • Maxent from OpenNLP v1.5 - $0
   • Perceptron from OpenNLP 1.5 - $0
   • NLTK Files from NLTK - $0
   • CDT Files from Copenhagen Treebank - $0
   • Penn Treebank 3 from LDC - $3000
5. Create development, library, and data directories:
   mkdir -p $HOME/dev/java/nlp/lib/
mkdir -p $HOME/dev/java/nlp/data/
6. Change to development directory:
   cd $HOME/dev/java/nlp/
7. Extract files:
   tar zxf $HOME/archives/apache-opennlp-*-incubating-src.tar.gz
8. Rename directory:
   mv apache-opennlp-*-incubating-src opennlp
9. Build Java Archive (JAR) files (takes about 6 minutes5 to 10 minutes, depending):
   cd opennlp/opennlp
mvn install > build.log
10. Change to OpenNLP development directory:
    cd $HOME/dev/java/nlp/opennlp/
11. Move library files to library directory:
    mv opennlp-uima/target/dependency/* ../lib/.
12. Move training data to data directory:
    mv $HOME/archives/pos/en-pos-maxent.bin $HOME/dev/java/nlp/data/.
13. Change to development directory:
    cd $HOME/dev/java/nlp/
14. Copy Download HelloWorld Source Code to $HOME/dev/java/nlp/HelloWorld.java.
15. Compile HelloWorld.java:
    javac -cp $(echo lib/*.jar | tr ' ' ':') HelloWorld.java
16. Run HelloWorld.java:
    java -cp .:$(echo lib/*.jar | tr ' ' ':') HelloWorld data/en-pos-maxent.bin "Earlier today, we compiled a program."

Output:

Earlier => JJR @ 0.2182545923597446
today, => NN @ 0.666361706870189
we => PRP @ 0.8324059729613176
compiled => VBN @ 0.028125261823754893
a => DT @ 0.9145975161653905
program. => NN @ 0.8841759649076423