Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleTopic Level Configs
public static final String REMOTE_COPY_LAG_MS_CONFIG = "remote.copy.lag.ms";
public static final String REMOTE_COPY_LAG_MS_DOC = "Controls how long to delay uploading segments to remote storage. " +
            "When set to 0, immediate upload when local time-based retention is used; otherwise no time-based without any delay check. " +
            "When set to a positive value (ms), a segment can't become eligible for upload until the time since the latest record in the segment reaches the value. " +
            "The value should not exceed the real local retention ms except the latter is retained indefinitely (-1). " +
            "When set to -1, resolves to the real local retention ms as maximum delay. " +
            "If the real local retention ms is configured as infinite, -1 is treated as anno invaliddelay configuration.check "on +
 this time configure. " +
            "For how the real local retention time is computed, see <code>local.retention.ms</code>.";

    public static final String REMOTE_COPY_LAG_BYTES_CONFIG = "remote.copy.lag.bytes";
    public static final String REMOTE_COPY_LAG_BYTES_DOC = "Controls size-based delay for uploading segments to remote storage. " +
            "When set to 0, immediate upload when local size-based retention is used; otherwise no size-basedwithout any delay check. " +
            "When set to a positive value (bytes), a segment can't become eligible for upload until the total bytes of log data after the segment reach the value. " +
            "The value should not exceed the real local retention bytes except the latter is retained indefinitely (-1). " +
            "When set to -1, resolves to the real local retention bytes as maximum delay. " +
            "If the real local retention bytes is configured as infinite, -1 is treated as an invalid configuration no delay check on this size configure. " +
            "For how the real local retention size is computed, see <code>local.retention.bytes</code>.";

...

We can use follow tests to cover the change:

Type\Configure

Case Id

local retention time

local retention size

lag time

lag size

Expect result

Notes

 

 

 

Special cases which can help to understand the basic rule for code review

 

 

1

-1

*

 *

0 (default value)

*

No lag
time check but check lag size if needWhy not upload at once?
if so, No way to get the size-based delay when only using the size-based retention.


2

*

 *

2

7 days

 *

0

*

No lagif have delay check on size. it may breack the time-based retention policy. so upload at once

3

-1

*

-1

*

Invalid configure

4

*

-1

*

0 (default value)

No
lag size check but check
lag
time if need
5

3

*

3 GB

-1

*

0

-1

No lag
 6
No delay check on size. check the time configure as final result

4

-1

*

-1

*

-1

No lag No delay check on time. check the size configure as final result
Invalid configure

 

 

Typical user cases which can test with deployment


 

7

5

 7 days


 

-1

 

0 (default value)

0 (default value)

No lag

7days + No size limit is
Kafka defalut configure

If you want time lag delay. you

only

need to set time

lag. no need to change

lag to real value and size lag to -1 

1 day

0 (default value)

-1

lag: 1 day

-1

0 (default value)

-1

lag: 7 days
8

6

 -1

3 GB

0 (default value)

0 (default value)

No lag 
0 (default value)

-1

1 GB

lag: 1 GB 
0 (default value)

-1

-1

lag: 3 GB 
9

7

7 days

3 GB

0 (default value)

0 (default value)

No lag 

1 day

1 GB

lag: 1 day or 1 GB 

-1

-1

lag: 7 days or 3 GB 
10

8

7 days

3 GB

1 day

0 (default value)

No lag
if check delay lag. it may break the size-based policy. So upload at once

Unit Tests:

  1. Test upload eligibility logic with different configure values
  2. Test configuration validation for topic

...