DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
public static final String REMOTE_COPY_LAG_MS_CONFIG = "remote.copy.lag.ms"; public static final String REMOTE_COPY_LAG_MS_DOC = "Controls how long to delay uploading segments to remote storage. " + "When set to -2 (default)0, noimmediate delayupload checkwithout basedany on local retention msdelay check. " + "When set to 0,a segmentspositive are uploaded as soon as they are eligible without any delay check. " + "When set to a positive value (msvalue (ms), a segment can't become eligible for upload utiluntil the time since the latest record in the segment reaches the value. " + "The value should not exceed the real local retention ms except the latter is retained indefinitely (-1). " + "When set to -1, resolves to the real local retention ms as maximum delay. " + "IfFor how the real local retention mstime is configured as infinite, -1 is treated as an invalid configuration. " + "For how the real local retention time is computed, computed, see <code>local.retention.ms</code>."; public static final String REMOTE_COPY_LAG_BYTES_CONFIG = "remote.copy.lag.bytes"; public static final String REMOTE_COPY_LAG_BYTES_DOC = "Controls size-based delay for uploading segments to remote storage. " + "When set to -2 (default)0, noimmediate delayupload checkwithout basedany on local retention bytesdelay check. " + "When set to 0,a segmentspositive are uploaded as soon as they arevalue (bytes), a segment can't become eligible withoutfor anyupload delayuntil check. " + "When set to a positive value (bytes), a segment can't become eligible for upload util the total bytes the total bytes of log data after the segment reach the value. " + "The value should not exceed the real local retention bytes except the latter is retained indefinitely (-1). " + "When set to -1, resolves to the real local retention bytes as maximum delay. " + "IfFor how the real local retention bytessize is configured as infinite, -1 is treated as an invalid configuration. " + "For how the real local retention size is computed, see <code>local.retention.bytes</code>."; |
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
public static final String LOG_REMOTE_COPY_LAG_MS_PROP = "log.remote.copy.lag.ms"; public static public static final String LOG_REMOTE_COPY_LAG_BYTES_PROP = "log.remote.copy.lag.bytes"; |
The default value are -2 0 so that the whole remote storage module keeps the original behavior.
...
We can use follow tests to cover the change:
Type\Configure | Case Id | local retention time | local retention size | lag time | lag size | Expect result | Notes |
Special cases which can help to understand the basic rule for code review
| 1 | * | * |
0 (default value) | * | No lag |
2 | * | * |
0
*
* | 0 (default value) | No lag | |
3 |
* | -1 | * |
-1 | No delay check on size. check the time configure as final result | For the case: 4 configures are -1. The final result is not upload. |
4 |
*
*
*
-2 (default value)
*
*
*
0
-1 | * | -1 |
* | No delay check on time. check the size configure as final result |
Typical user cases which can test with deployment
|
5 | 7 days
| -1
|
0 (default value) |
0 (default value) | No lag | 7days + No size limit is If you want time lag delay. you need to set time lag to real value and size lag to -1 |
1 day | - |
1 | lag: 1 day |
-1 | - |
1 | lag: |
| 7 days | ||
6 | -1 | 3 GB |
0 (default value) |
0 (default value) | No lag | |
- |
1 | 1 GB | lag: 1 GB | |
- |
1 | -1 | lag: 3 GB | |
7 |
7 days | 3 GB |
0 (default value) |
0 (default value) | No lag | |
1 day | 1 GB | lag: 1 |
| day or 1 GB | ||
-1 | -1 | lag: 7 days or 3 |
| GB | ||||||
8 | 7 days | 3 GB | 1 day | 0 (default value) | No lag |
Unit Tests:
- Test upload eligibility logic with different configure values
- Test configuration validation for topic
...