THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
This SystemProducer allows the following properties to be configurable.
...
| Name | Default | Description |
|---|
| systems.<azure-container-name>.azureblob.account.name |
...
| Azure account |
...
| name. Is not optional |
...
| systems.%s.azureblob.account.key |
...
| Azure account key. Is not optional |
| systems.%s.azureblob.proxy.use |
...
| "false |
...
| " | if true, proxy will be used to connect to Azure. |
| systems.%s.azureblob.proxy.hostname |
...
| if proxy.use is true then host name of proxy | |
| systems.%s.azureblob.proxy.port |
...
| if proxy.use is true then port of proxy |
| systems.%s.azureblob.compression.type |
...
| "none" |
...
| type of compression to be used before uploading blocks. Can be "none" or "gzip" |
| systems.%s.azureblob.maxFlushThresholdSize |
...
| 10485760 (10 MB) |
...
| max size of the uncompressed |
...
| block to be uploaded in bytes |
...
| . Maximum size allowed by Azure is 100MB. |
| systems.%s.azureblob.maxBlobSize |
...
| Long.MAX_VALUE - unlimited |
...
| max size of the uncompressed blob in bytes. If default value then size is unlimited capped only by Azure BlockBlob size of 4.75 TB (100 MB per block X 50,000 blocks) |
...
| . |
| systems.%s.azureblob.maxMessagesPerBlob |
...
| Long.MAX_VALUE - unlimited | max number of messages per blob | |
| systems.%s.azureblob.threadPoolCount | 2 | number of threads for the asynchronous uploading of blocks |
...
systems.%s.azureblob. |
...
blockingQueueSize | Thread Pool Count * 2 | size of the queue to hold blocks ready to be uploaded by asynchronous threads. If all threads are busy uploading then blocks are queued and if queue is full then main thread will start uploading which will block processing of incoming messages |
| systems.%s.azureblob.flushTimeoutMs |
...
| 3 mins | timeout to finish uploading all blocks before committing a blob | |
| systems.%s.azureblob.closeTimeoutMs | 5 mins | timeout to finish committing all the blobs currently being written to. This does not include the flush timeout per blob |
| systems.%s.azureblob. |
...
- default: 5 mins
...
| suffixRandomStringToBlobName | false | if true, a random string |
...
| of 8 chars is suffixed to the blob name to |
...
| prevent name collision when more than one Samza tasks are writing to the same SSP. It is advisable to set this to true |
Implementation / Test Plan
...