Page History

...

- Java class Replica will keep track of two instances of Java class Log, one representing topicPartition.log and the other representing topicPartition.move
- A ReplicaFetchThread will push data from topicPartition.log to topicPartition.move. If the Replica is a follower, then the thread will be the same ReplicaFetchThread which is appending data to topicPartition.log. If the Replica is a leader, ReplicaFetcherManager will add fetcher for this topicPartition. Every time this thread executes create a new ReplicaFetchThread to move replicas between its own disks. We will need quota to rate limit replica movement if the replica is a leader. 
- When ReplicaFetchThread builds FetchRequset, it sets maximum wait time to 0 ms if it needs to move any replica between its own disks.
- When ReplicaFetchThread executes processPartitionData(topicPartition) after receiving FetchResponse, it will check if the local replica of this topicPartition has Log instances for both topicPartition.log and topicPartition.move. If yes, it will read a ByteBufferMessageSetone ByteBufferMessageSet of size replica.fetch.max.bytes from topicPartition.log and append the message set to topicPartition.move.
- Replace topicPartition.log with topicPartition.move when topicPartition.move has caught up.
  - If the Replica is a follower, and if topicPartition.move has caught up with topicPartition.log, the ReplicaFetchThread which is appending data to this Replica will rename topicPartition.log to topicPartition.delete, rename topicPartition.move to topicPartition.log, and update the Replica instance of this topicPartition to only include a Log instance that points to the new topicPartition.log. The data from leader will be appended to the new log file in the future.
  - If the Replica is a leader, and if topicPartition.move has caught up with topicPartition.log,  the KafkaRequestHandler thread which is handling a ProduceRequest to this Replica will rename topicPartition.log to topicPartition.delete, rename topicPartition.move to topicPartition.log, update the Replica instance of this topicPartition to only include a Log instance that points to the new topicPartition.log, and append data from ProduceRequest to the new log file.

2. The broker logic for handling ChangeReplicaDirRequest

...