Clinical Documents from an EMR are typically stored in a database. We developed an UIMA CollectionReader that retrieves these documents for annotation (thereby avoiding exporting the documents to the file system). The DBCollection reader works as follows:
Documents in a database typically are given a Unique ID, and are cross-referenced with other information (e.g. patient id, document type, document date, etc.). The YTEX DBConsumer can store the unique id in the database along with other document annotations. This allows you to cross-reference annotation data with other document data.
By default, YTEX supports an integer document identifier (INSTANCE_ID) and/or a string document identifier (INSTANCE_KEY). When using the FileSystemCollection reader, the file name is stored in the INSTANCE_KEY.
The YTEX DBCollection reader is parameterized by 2 queries: a key query and a document query. The key query loads document unique ids from a database, and the document query loads a document for a given unique id. For more information, refer to the example below.
Note that you must perform the additional YTEX installation tasks to use this component.
To illustrate this, we can simply retrieve sample documents stored in the fracture_demo table. To configure and execute the CPE, do the following:
MS SQL Server
select note_id instance_id from <schema>.fracture_demo
select note_text from <schema>.fracture_demo where note_id = :instance_id
MySQL
select note_id instance_id from fracture_demo
select note_text from fracture_demo where note_id = :instance_id
Oracle
select note_id instance_id from fracture_demo
select note_text from fracture_demo where note_id = :instance_id
Click on the 'Add' button in the 'Analysis Engine' section, and select CTAKES_HOME\desc\ytex-uima\desc\analysis_engine\AggregatePlaintextUMLSProcessor.xml
select * from
v_document where analysis_batch = 'test2'
Notice that the instance_id column is set, and refers to the fracture_demo.note_id.
This example demonstrates how to use custom document keys. Let's assume that clinical documents have a unique identifier that comprises 2 fields - note_id (integer) and site_id (character) - and that these documents are stored in the fracture_demo table. We would like to link our annotations to the original document, so we need to store both the note_id and site_id in the ytex document table. In this example, we map these columns to the instance_id and site_id columns in the document table. (We could map the site_id to INSTANCE_KEY, but that would be no fun.)
To run this example, do the following:
alter table document add site_id varchar(20)
Windows: Run YTEX_HOME/ytexCPE.cmd Unix: from a shell run the following commands
. ${HOME}/ytex.profile
cd ${YTEX_HOME}
java ${JAVA_OPTS} org.apache.uima.tools.cpm.CpmFrame
Go to File->Open, and select YTEX_HOME/examples/cpe-fracture/fracture-demo.cpe.xml. The query to get the keys looks like this (notice how note_id was renamed to instance_id):
select note_id instance_id, site_id from fracture_demo
The query to get the document looks like this:
select note_text from fracture_demo where note_id = :instance_id and site_id = :site_id
The CPE config "Store Doc Text" checkbox is unchecked: we will note store the document text in the document.doc_text column, because it is already in the database - we can join the document and fracture_demo tables on the uid/site_id columns to get the corresponding text.
select * from document where analysis_batch = 'cpe-fracture'