DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Our proposal is in a way also trying to reduce the state size of Kafka broker by uploading more recent log segments onto object storage but we are implementing it in a more conservative way.
Note that we mentioned KIP-1150 in the Rejected Alternative section but we don't view this KIP and KIP-1150 are all-or-nothing alternatives, they are optimized for different use cases (this KIP proposal are optimizing for low-latency acks=1 use case while KIP-1150 is optimizing for a longer latency but diskless implementation), we believe some of the ideas or implementations can be learned or adopted by both KIPs.
Appendix: Cost Estimate for a typical workload
...
See the following table for cost comparison. The charge is per month per kafka broker
Baseline | S3E1Z | |
Storage/Instance Cost | $866 | $776 |
Transfer Cost | $1612 | $648 |
Total Cost | $2478 | $1424 |
Latency | Single digit ms | Single digit ms |
Across AZ | Yes | Yes |
The result is ~43% reduction in cost.
...