...
- as a consumer, if it's a file, it just reads the file, otherwise if it represents a directory it scans all the file under the path satisfying the configured pattern. All the files under that directory must be of the same type.
- as a producer, if at least one split strategy is defined, the path is considered a directory and under that directory the producer creates a different file per split named using the configured UuidGenerator.
Options
| Div |
|---|
| class | confluenceTableSmall |
|---|
|
Name | Default Value | Description |
|---|
overwrite | true | The file can be overwritten | append | false | Append to existing file. Notice that not all HDFS file systems support the append option. | bufferSize | 4096 | The buffer size used by HDFS | replication | 3 | The HDFS replication factor | blockSize | 67108864 | The size of the HDFS blocks | fileType | NORMAL_FILE | It can be SEQUENCE_FILE, MAP_FILE, ARRAY_FILE, or BLOOMMAP_FILE, see Hadoop | fileSystemType | HDFS | It can be LOCAL for local filesystem | keyType | NULL | The type for the key in case of sequence or map files. See below. | valueType | TEXT | The type for the key in case of sequence or map files. See below. | splitStrategy | | A string describing the strategy on how to split the file based on different criteria. See below. | openedSuffix | opened | When a file is opened for reading/writing the file is renamed with this suffix to avoid to read it during the writing phase. | readSuffix | read | Once the file has been read is renamed with this suffix to avoid to read it again. | initialDelay | 0 | For the consumer, how much to wait (milliseconds) before to start scanning the directory. | delay | 0 | The interval (milliseconds) between the directory scans. | pattern | * | The pattern used for scanning the directory | chunkSize | 4096 | When reading a normal file, this is split into chunks producing a message per chunk. | connectOnStartup | true | Camel |
|
| Wiki Markup |
|---|
| {div:class=confluenceTableSmall}
|| Name || Default Value || Description ||
| {{overwrite}} | {{true}} | The file can be overwritten |
| {{append}} | {{false}} | Append to existing file. Notice that not all HDFS file systems support the append option. |
| {{bufferSize}} | {{4096}} | The buffer size used by HDFS |
| {{replication}} | {{3}} | The HDFS replication factor |
| {{blockSize}} | {{67108864}} | The size of the HDFS blocks |
| {{fileType}} | {{NORMAL_FILE}} | It can be SEQUENCE_FILE, MAP_FILE, ARRAY_FILE, or BLOOMMAP_FILE, see Hadoop |
| {{fileSystemType}} | {{HDFS}} | It can be LOCAL for local filesystem |
| {{keyType}} | {{NULL}} | The type for the key in case of sequence or map files. See below. |
| {{valueType}} | {{TEXT}} | The type for the key in case of sequence or map files. See below. |
| {{splitStrategy}} | | A string describing the strategy on how to split the file based on different criteria. See below. |
| {{openedSuffix}} | {{opened}} | When a file is opened for reading/writing the file is renamed with this suffix to avoid to read it during the writing phase. |
| {{readSuffix}} | {{read}} | Once the file has been read is renamed with this suffix to avoid to read it again. |
| {{initialDelay}} | {{0}} | For the consumer, how much to wait (milliseconds) before to start scanning the directory. |
| {{delay}} | {{0}} | The interval (milliseconds) between the directory scans. |
| {{pattern}} | {{*}} | The pattern used for scanning the directory |
| {{chunkSize}} | {{4096}} | When reading a normal file, this is split into chunks producing a message per chunk. |
| {{connectOnStartup}} | {{true}} | *Camel * {{}} {{}} |
| {{owner}} | | *Camel * |
{div} |
KeyType and ValueType
- NULL it means that the key or the value is absent
- BYTE for writing a byte, the java Byte class is mapped into a BYTE
- BYTES for writing a sequence of bytes. It maps the java ByteBuffer class
- INT for writing java integer
- FLOAT for writing java float
- LONG for writing java long
- DOUBLE for writing java double
- TEXT for writing java strings
...
The following headers are supported by this component:
Producer only
| Div |
|---|
| class | confluenceTableSmall |
|---|
|
Header | Description |
|---|
CamelFileName | Camel |
|
| Wiki Markup |
|---|
| {div:class=confluenceTableSmall}
|| Header || Description ||
| {{CamelFileName}} | *Camel * {{}} [] |
{div} |
Controlling to close file stream
...
