THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
org.apache.oodt.cas.pushpull.exceptions.RemoteConnectionException:
[ProtocolHandler]
Packet corruption
$ tc qdisc change dev eth0 root netem corrupt 1%
The packet corruption is not seen to have a significant impact on the performance of file transfer for packet corruption < 8%. For packets > 8% being corrupted, the file transfer time is seen to increase. For packet corruption > 32%, the terminal itself became unresponsive and incapable of testing file transfer using PushPull.
Packet reordering
$ tc qdisc change dev eth0 root netem delay 100ms reorder 1% 50%
This argument implies that 1% of packets sent will be reordered and delayed by 100ms with the next packet having a probability of 0.5 of also being reordered. The packet delay as well as the next packet being reordered probability were kept constant throughout the evaluation.
Packet reordering was found to significantly increase the file transfer time for packet corruption > 16% with a very surprising decrease in the transfer time for 32%. This behaviour was noticed for several test runs using the same parameter set.
Packet duplication
$ tc qdisc change dev eth0 root netem duplicate 1%
Although not a linear nor consistent increase in transfer time was found for packet duplication, an overall increase is marked. PushPull was found capable of dealing with great levels of packet duplication without stalling or terminating file transfer.
Modified TCP window
A single file of 1024MB was downloaded using PushPull using different TCP window sizes. There is the option to enable/disable TCP window scaling in the configuration file /etc/sysctl.conf and to set the maximum transfer window under LINUX. The file is then reloaded to update the parameters using
$ sysctl -p
The parameters in /etc/sysctl.conf which were modified are (e.g.):
...
| Window size [Bytes] | Transfer Time [sec] |
| 128 | ERROR |
| 256 | 91.13 |
| 512 | 86.56 |
| 1024 | 71.2 |
| 2048 | 51.41 |
| 4096 | 51.59 |
| 8192 | 50.43 |
| 16384 | 50.97 |
Concurrent file transfer
Modifying OODT thread number
The performance of OODT PushPull was evaluated for directory transfer using a directory structure of 16x1024MB binary files. This would result in an averaged outcome compared to single file transfer and also eliminating some anomalies which arose in some cases.
PushPull has the option of specifying the number of threads for download of multiple files in parallel e.g (/caspushpull0.6/etc/push_pull_framework.properties):
org.apache.oodt.cas.pushpull.file.retrieval.system.recommended.thread.count=4
which will be used when the optimised variable is set to false:
org.apache.oodt.cas.pushpull.crawler.use.tracker=false
By manually setting the number of threads for concurrent file transfer, the optimised thread number was found to be 4 which was therefore used for further benchmarking of concurrent file transfer on a modified network.
Concurrent file transfer with packet delay
Using the recommended/optimised thread number for concurrent file transfer, the performance of PushPull is evaluated with induced packet delay into the network. While the bandwidth decreases for increasing induced packet delay, as expected, the tool is still capable of transferring files concurrently even with packet delays of ~1s (1000ms).
Concurrent file transfer with packet loss
Little or no performance variation was found for packet loss of < 4%. When a large percentage of packet loss was induced, > 16%, the transfer time was exponentially increased up to a point ( > 24%) where the tool was not capable of transferring.
Concurrent file transfer with packet corruption
The performance of PushPull was found to drop to ~0.5 of the “clean network link” for a packet corruption = 24%. Beyond this point PushPull was not able to transfer files as the tool became unresponsive.
Concurrent file transfer with packet reordering
A peculiar, non-linear, behaviour was found when inducing packet reordering < 4% for concurrent file transfer. These tests were once again repeated but still confirmed this behaviour. In particular from 1 -2% an increase is noted in the transfer rate. For packet reordering > 8% a general decreasing trend is noted for the transfer rate. While the transfer rate decreases, it still remains > 30 MB/s indicating that the tool manages packet reordering well with very little variation (or decrease) in performance observed, even for reordering > 80%.
Concurrent file transfer with packet duplication
Packet duplication shows a strong linear trend. Although a decrease in the transfer rate is seen, it is only ~3 MB/s going from 1-80%. . Therefore, PushPull is more than capable to deal with packet duplication.
