Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Data Files

...

Introduction

There is a data import tool in OFBiz called the DataFile tool. It uses XML files that describe flat file formats (including character delimited, fixed width, etc) and parses the flat files based on those definitions. So, by design it is somewhat like the Entity Engine. It uses a generic object to represent a row in the flat file. It includes features like a line type code for each line and can support hierarchical flat files (ie where parent/child relationships are implied by sub-records).

...

  • the record.name attribute must contain the name of the entity in which the records will be imported (for example: <record name="Product" limit="many">)
  • the field.name attribute must contain the name of the entity field in which the records will be imported (for example: <field name="productId" type="String"/>)

Examples

An example of fixed width flat file import.

Sample fixed width CSV file posreport.csv to be imported:

Code Block

021196033702    ,5031BB GLITTER GLUE PENS BRIGH  ,1           ,5031BB      ,       1,     299,
021196043121    ,BB4312 WONDERFOAM ASSORTED      ,1           ,BB4312      ,       1,     280,
021196055025    ,9905BB  PLUMAGE MULTICOLOURED   ,1           ,9905BB      ,       4,     396,




Sample xml definition file for importing select columns posschema.xml:

Code Block

<?xml version="1.0" encoding="UTF-8" ?>

<data-files xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ofbiz.apache.org/dtds/datafiles.xsd">
    <data-file name="posreport" separator-style="fixed-length" type-code="text">
        <record name="tillentry" limit="many">
            <field name="tillCode" type="String" length="16" position="0"/>
            <field name="name" type="String" length="32" position="17"/>
            <field name="prodCode" type="String" length="12" position="63"/>
            <field name="quantity" type="String" length="8" position="76"/>
            <field name="totalPrice" type="String" length="8" position="85"/>
        </record>
    </data-file>
</data-files>




In the interface enter something like:

  • Definition Filename or URL: posschema.xml
  • Data File Definition Name: posreport
  • Data Filename or URL: posreport.csv

The types listed in this sample are simple String's but the usual types are available such as Date, Long etc.

Another example reading fixed record little endian binary files


Code Block

<?xml version="1.0" encoding="UTF-8" ?>

<data-files xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ofbiz.apache.org/dtds/datafiles.xsd">
    <data-file name="stockdata" separator-style="fixed-record" type-code="text" record-length="768">
        <record name="stockdataitem" limit="many">
            <field name="barcode" type="NullTerminatedString" length="12" position="0"/>
            <field name="prodCode" type="NullTerminatedString" length="12" position="68"/>
            <field name="price" type="LEInteger" length="4" position="80"/>
            <field name="name" type="NullTerminatedString" length="30" position="16"/>
        </record>
    </data-file>
</data-files>




Tab delimited imports

Worth noting that to get tab separated imports working you should use a delimiter of : 

Code Block

<?xml version="1.0" encoding="UTF-8"?>
<data-files xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://ofbiz.apache.org/dtds/datafiles.xsd">
    <data-file name="bacs-iscd" separator-style="delimited" type-code="text" delimiter="&#009;">........


Also you may have a look at real life examples in
applications\order\src\org\ofbiz\order\thirdparty\zipsales\ZipSalesServices.java and
applications\order\src\org\ofbiz\order\thirdparty\taxwareYou can find some examples in this page.