THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Master branch build time: | 06:52 22 min |
|---|---|
| Feature branch HDDS-4440-s3-performance build time: | 06:16 min |
...
10. third party dependencies/license changes:
For the s3-performance Grpc feature, network transport related jars are added to support native encryption on the wire, TLS:
Added to License.txt |
|---|
| + io.netty:netty-tcnative-boringssl-static |
| + io.netty:netty-tcnative |
11. performance
We compare the performance of the s3 gateway using the Grpc persistent connection with TLS to the existing Hadoop RPC, hRPC connections with encryption on the wire for metadata requests. We find that in load testing the s3 performance feature branch with Grpc and encryption on the wire outperforms the existing hRPC connection both encrypted and in plaintext. This is particularly evident in the comparison of Grpc with TLS to encrypted wire Hadoop RPC where the increase is greater than 2X.
| s3g Transport Type | Description | Load Test Performance for Metadata throughput, Objects / sec (objs/sec) | |
|---|---|---|---|
| 1 | Grpc TLS (feature branch) | s3g ↔ ozone manager connection over Grpc with encryption on the wire, TLS. Persistent connection. | 9026.12 |
| 2 | hRPC plaintext (current) | s3g ↔ ozone manager connection over Hadoop Rpc plaintext. Persistent connection (HDDS-5881). | 6508.85 |
| 3 | hRPC encrypted wire (current) | s3g↔ ozone manager connection over Hadoop Rpc with encryption on the wire (privacy configuration). Persistent connection (HDDS-5881). | 3989.35 |
Load test used: minio Warp S3 benchmarking tool.
| Wiki Markup |
|---|
{$ ./warp stat --host=<host> --duration=1s –bucket bucket1 --concurrent=64 --noclear --obj.size=1KiB --access-key=$AWS_ACCESS_KEY --secret-key=$AWS_SECRET_ACCESS_KEY} |
12. security considerations
...