...

• Make the below configs in the RemoteLogManager to be updated dynamically. Once they are updated, the respective thread-pool will be resized:
  • remote.log.manager.copier.thread.pool.size
  • remote.log.manager.expiration.thread.pool.size 
  • remote.log.reader.threads and
  • remote.log.manager.thread.pool.size (deprecated in KIP-950, we can skip making this config as dynamic)
• Add a new method isInitialized in isReady in RemoteLogMetadataManager.

...

1. The thread-pools used by the RemoteLogManager does not allow to reconfigure the thread count dynamically. This is similar to updating the request handler thread counts dynamically and required to handle any unprecedented load in the remote storage. 
2. When a broker comes up online, the RemoteLogMetadataManager (RLMM) gets initialized in the background using the async fashion. The server starts to copy/delete the expired remote log segments, then the RLMM throws exception until it gets initialized. By adding the isInitialized isReady method, the server can delay the copying (or) deleting the remote log segments until the RLMM gets initialized. This is a graceful handling to avoid exception/error logs while starting the server. 

...

    /**
     * Denotes whether the partition metadata is ready to serve.
     *
     * @param topicIdPartition topic partition
     * @return True if the partition is initializedready to serve for remote storage operations.
     */
    default boolean isReady(TopicIdPartition topicIdPartition) {
        return true;
    }

...

• Unit test to assert that the remote-log manager configs can be updated dynamically and thread-pool gets resized. 
• Unit test to assert that the copy/expiration logic won't be invoked for a partition until the RLMM gets initialized for itready to serve remote storage operations. 

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.