Test Plan for Apache OpenNLP 1.5.3

This page contains the test plan for the 1.5.3 release.

The 1.5.3 release does not introduce any changes to the feature
generation.

Compatibility Test with OpenNLP 1.5.0 SourceForge Models

The 1.5.0 SourceForge models must be fully compatible with the 1.5.3
release. In this test all the English models are tested for compatibility
on the English 300k sentences Leipzig Corpus. It is tested that
the output produced with the same model by both versions has the same md5 hash.

Component	Model	Perf 1.5.2	Perf 1.5.3	Tester	Passed	Comment
Sentence Detector	en-sent.bin
Tokenizer	en-token.bin
Name Finder	en-ner-person.bin
POS Tagger	en-pos-maxent.bin
POS Tagger	en-pos-perceptron.bin
Chunker	en-chunker.bin
Parser	en-parser-chunking.bin

Note: Test was done on Hardware running Operational System
and Java Java Version. The performance varies because light weight tasks have been performed in the background while testing.

Note: "Concurrent" in the comment means that both tests where started at the same time.

Regression Test Training with (private) English data

The training of both versions with the same data must produce
a model with identical output. The model output is tested with
the procedure from the previous test.

To pass the test the event hash and the model output must be identical.

Component	Model	Training Time 1.5.2	Training Time 1.5.3	Tester	Passed	Comment
Sentence Detector	en-sent.bin
Tokenizer	en-token.bin
POS Tagger	en-pos-maxent.bin
POS Tagger	en-pos-perceptron.bin
Parser	en-parser-chunking.bin

Note: Time was measured with the time command, the value is the "real" time value.

Performance test with public data

Test the tagging performance with all the publicly available training
and test data for various languages.

It is assumed that the training will be done with a cutoff of 5 and 100 iterations,
if different values are used please write them into the comment.

Component	Data	Tagging Perf 1.5.2	Comment
Sentence Detector
Tokenizer
Name Finder	CONLL 2002 Dutch Person ned.testa	Precision: 0.7552941176470588 Recall: 0.4566145092460882 F-Measure: 0.5691489361702128
Name Finder	CONLL 2002 Dutch Person ned.testb	Precision: 0.8505025125628141 Recall: 0.6165755919854281 F-Measure: 0.7148891235480465
Name Finder	CONLL 2002 Dutch Organization ned.testa	Precision: 0.8561872909698997 Recall: 0.37317784256559766 F-Measure: 0.5197969543147207
Name Finder	CONLL 2002 Dutch Organization ned.testb	Precision: 0.7830374753451677 Recall: 0.4501133786848073 F-Measure: 0.5716342692584593
Name Finder	CONLL 2002 Dutch Location ned.testa	Precision: 0.8458333333333333 Recall: 0.42379958246346555 F-Measure: 0.564673157162726
Name Finder	CONLL 2002 Dutch Location ned.testb	Precision: 0.8816326530612245 Recall: 0.5581395348837209 F-Measure: 0.6835443037974683
Name Finder	CONLL 2002 Dutch Misc ned.testa	Precision: 0.8354114713216958 Recall: 0.44786096256684493 F-Measure: 0.5831157528285466
Name Finder	CONLL 2002 Dutch Misc ned.testb	Precision: 0.8264984227129337 Recall: 0.44144903117101936 F-Measure: 0.5755079626578803
Name Finder	CONLL 2002 Combined ned.testa	Precision: 0.6509695290858726 Recall: 0.628822629969419 F-Measure: 0.6397044526540929	1000 iterations
Name Finder	CONLL 2002 Dutch Combined ned.testb	Precision: 0.6869929337869668 Recall: 0.6660746003552398 F-Measure: 0.6763720690543674	1000 iterations
Name Finder	CONLL 2002 Spanish Person esp.testa	Precision: 0.9010695187165776 Recall: 0.5515548281505729 F-Measure: 0.684263959390863
Name Finder	CONLL 2002 Spanish Person esp.testb	Precision: 0.9195205479452054 Recall: 0.7306122448979592 F-Measure: 0.8142532221379833
Name Finder	CONLL 2002 Spanish Organization esp.testa	Precision: 0.8288942695722357 Recall: 0.6041176470588235 F-Measure: 0.6988771691051379
Name Finder	CONLL 2002 Spanish Organization esp.testb	Precision: 0.8036277602523659 Recall: 0.7278571428571429 F-Measure: 0.7638680659670164
Name Finder	CONLL 2002 Spanish Location esp.testa	Precision: 0.7743016759776536 Recall: 0.7042682926829268 F-Measure: 0.7376263970196913
Name Finder	CONLL 2002 Spanish Location esp.testb	Precision: 0.8301886792452831 Recall: 0.5682656826568265 F-Measure: 0.6746987951807228
Name Finder	CONLL 2002 Spanish Misc esp.testa	Precision: 0.6492890995260664 Recall: 0.30786516853932583 F-Measure: 0.4176829268292683
Name Finder	CONLL 2002 Spanish Misc esp.testb	Precision: 0.686046511627907 Recall: 0.3480825958702065 F-Measure: 0.461839530332681
Name Finder	CONLL 2002 Spanish Combined esp.testa	Precision: 0.7005423249233671 Recall: 0.6828315329809239 F-Measure: 0.6915735567970205	1000 iterations
Name Finder	CONLL 2002 Spanish Combined esp.testb	Precision: 0.756635931824532 Recall: 0.7611017425519955 F-Measure: 0.7588622670589884	1000 iterations
Name Finder	CONLL 2003 English Person eng.testa	Precision: 0.9523195876288659 Recall: 0.8023887079261672 F-Measure: 0.8709487330583382
Name Finder	CONLL 2003 English Person eng.testb	Precision: 0.9391727493917275 Recall: 0.7161410018552876 F-Measure: 0.8126315789473685
Name Finder	CONLL 2003 English Organization eng.testa	Precision: 0.8768046198267565 Recall: 0.6793437733035048 F-Measure: 0.7655462184873949
Name Finder	CONLL 2003 English Organization eng.testb	Precision: 0.8435980551053485 Recall: 0.6267308850090307 F-Measure: 0.7191709844559586
Name Finder	CONLL 2003 English Location eng.testa	Precision: 0.9361421988150099 Recall: 0.7740881872618399 F-Measure: 0.8474374255065554
Name Finder	CONLL 2003 English Location eng.testb	Precision: 0.9206349206349206 Recall: 0.7302158273381295 F-Measure: 0.8144433299899699
Name Finder	CONLL 2003 English Misc eng.testa	Precision: 0.9027982326951399 Recall: 0.6648590021691974 F-Measure: 0.7657713928794503
Name Finder	CONLL 2003 English Misc eng.testb	Precision: 0.8592436974789915 Recall: 0.5826210826210826 F-Measure: 0.6943972835314092
Name Finder	CONLL 2003 English Combined eng.testa	Precision: 0.861812521618817 Recall: 0.8386065297879501 F-Measure: 0.8500511770726714	1000 iterations
Name Finder	CONLL 2003 English Combined eng.testb	Precision: 0.8041311831853597 Recall: 0.7857648725212465 F-Measure: 0.7948419450165667	1000 iterations
Name Finder	CONLL 2003 German Person deu.testa	Precision: 0.9132653061224489 Recall: 0.25553176302640973 F-Measure: 0.3993307306190742
Name Finder	CONLL 2003 German Person deu.testb	Precision: 0.8732106339468303 Recall: 0.3573221757322176 F-Measure: 0.507125890736342
Name Finder	CONLL 2003 German Organization deu.testa	Precision: 0.8407224958949097 Recall: 0.4125705076551168 F-Measure: 0.5535135135135135
Name Finder	CONLL 2003 German Organization deu.testb	Precision: 0.8014705882352942 Recall: 0.4230271668822768 F-Measure: 0.5537679932260795
Name Finder	CONLL 2003 German Location deu.testa	Precision: 0.7816326530612245 Recall: 0.32430143945808637 F-Measure: 0.45840813883901854
Name Finder	CONLL 2003 German Location deu.testb	Precision: 0.8033826638477801 Recall: 0.3671497584541063 F-Measure: 0.5039787798408487
Name Finder	CONLL 2003 German Misc deu.testa	Precision: 0.7055555555555556 Recall: 0.12574257425742574 F-Measure: 0.21344537815126052
Name Finder	CONLL 2003 German Misc deu.testb	Precision: 0.6601307189542484 Recall: 0.15074626865671642 F-Measure: 0.2454434993924666
Name Finder	CONLL 2003 German Combined deu.testa	Precision: 0.7718859429714857 Recall: 0.319263397475688 F-Measure: 0.4516978922716628
Name Finder	CONLL 2003 German Combined deu.testb	Precision: 0.7467566165023353 Recall: 0.3917778382793357 F-Measure: 0.5139285714285715
POS Tagger	CONLL 2006 Danish	Accuracy: 0.9511278195488722
POS Tagger	CONLL 2006 Dutch	Accuracy: 0.9324977618621307
POS Tagger	CONLL 2006 Portuguese	Accuracy: 0.9659110277825124
POS Tagger	CONLL 2006 Swedish	Accuracy: 0.9275106082036775
Chunker	CONLL 2000	Precision: 0.9257575757575758 Recall: 0.9221868187154117 F-Measure: 0.9239687473746113
Chunker	Arvores Deitadas (10-fold cross-validation)	Precision: 0.9403445830378374 Recall: 0.9373141775994345 F-Measure: 0.9388269348910339

Test UIMA Integration

The test ensures that the Analysis Engine can run and not not
crash trough simple runtime time code errors. We need to add
more sophisticated testing with the next releases.

Analysis Engine	Tester	Passed	Comment
Sentence Detector
Sentence Detector Trainer
Tokenizer ME
Tokenizer Trainer
Name Finder
Name Finder Trainer
Chunker
Chunker Trainer
POS Tagger
POS Tagger Trainer
Parser
createPear.sh
Sample PEAR

Distribution Review

Please ensure that the listed files below are included in the distributions
and are in a good state.

Package	File or Test	Comment
Binary	LICENSE	AL 2.0 and BSD for JWNL
Binary	NOTICE	standard notice, dates are correct. JWNL is mentioned
Binary	README	File was reviewed on the dev list.
Binary	RELEASE_NOTES.html	issue list is generated correctly
Binary	Test signatures: .md5, .sha1, .asc	rc4
Binary	JIRA issue list created
Binary	Contains maxent, tools, uima and jwnl jars
Source	LICENSE	standard AL 2.0 file
Source	NOTICE	standard notice, dates are correct
Source	Test signatures: .md5, .sha1, .asc	rc4
Source	Can build from source?	Test should be done without jwnl and opennlp in local m2 repo. Test was done on Ubuntu 10.10.

Child pages

TestPlan1.5.3