Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleTopic Level Configs
public static final String REMOTE_COPY_LAG_MS_CONFIG = "remote.copy.lag.ms";
public static final String REMOTE_COPY_LAG_MS_DOC = "Controls how long to delay uploading segments to remote storage. " +
            "When set to -2 (default), no delay check based on local retention ms. " +
            "When set to 0, segments are uploaded as soon as they are eligible without any 0, immediate upload when local time-based retention is used; otherwise no time-based delay check. " +
            "When set to a positive value (ms), a segment can't become eligible for upload utiluntil the time since the latest record in the segment reaches the value. " +
            "The value should not exceed the real local retention ms except the latter is retained indefinitely (-1). " +
            "When set to -1, resolves to the real local retention ms as maximum delay. " +
            "If the real local retention ms is configured as infinite, -1 is treated as an invalid configuration. " +
            "For how the real local retention time is computed, see <code>local.retention.ms</code>.";

public static final String REMOTE_COPY_LAG_BYTES_CONFIG = "remote.copy.lag.bytes";
public static final String REMOTE_COPY_LAG_BYTES_DOC = "Controls size-based delay for uploading segments to remote storage. " +
            "When set to -2 (default), no delay check based on local retention bytes "Controls size-based delay for uploading segments to remote storage. " +
            "When set to 0, segmentsimmediate areupload uploadedwhen aslocal soonsize-based asretention theyis areused; eligibleotherwise withoutno anysize-based delay check. " +
            "When set to a positive value (bytes), a segment can't become eligible for upload utiluntil the total bytes of log data after the segment reach the value. " +
            "The value should not exceed the real local retention bytes except the latter is retained indefinitely (-1). " +
            "When set to -1, resolves to the real local retention bytes as maximum delay. " +
            "If the real local retention bytes is configured as infinite, -1 is treated as an invalid configuration. " +
            "For how the real local retention size is computed, see <code>local.retention.bytes</code>.";

...

Code Block
languagejava
titleBroker Level Configs
    public static final String LOG_REMOTE_COPY_LAG_MS_PROP = "log.remote.copy.lag.ms";
    public static final String LOG_REMOTE_COPY_LAG_BYTES_PROP = "log.remote.copy.lag.bytes";


The default value are -2 0 so that the whole remote storage module keeps the original behavior.

...

We can use follow tests to cover the change:

Type\Configure

local retention time

local retention size

lag time

lag size

Expect result

 

 

 

Special cases which can help to understand the basic rule for code review

 

 

*

-1

 *

-2

0 (default value)

*

No lag time check but check lag size if need
*

3 days

 *

0

*

No
any delay check. 
lag

-1

*

-1

*

Invalid configure

*

*

-1

*

-2

0 (default value)

No lag size check but check lag time if need

*

*

3 GB

*

0

No
any delay check. 
lag

*

-1

*

-1

Invalid configure

 

 

Typical user cases which can test with deployment

 3 days


 

-1

 

-2

0 (default value)

-2

0 (default value)

No lag

1 day

-2

0 (default value)

lag: 1 day

-1

-2

0 (default value)

lag:  3 days

 -1

3 GB

-2

0 (default value)

-2

0 (default value)

No lag
-2

0 (default value)

1 GB

lag: 1 GB
-2

0 (default value)

-1

lag: 3 GB

3 days

3 GB

-2

0 (default value)

-2

0 (default value)

No lag

1 day

1 GB

lag: 1 days or 1 GB

-1

-1

lag: 3 days or 3 GB

Unit Tests:

  1. Test upload eligibility logic with different configure values
  2. Test configuration validation for topic

...