Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleRateLimitingStrategy
/**
 * RateLimitingStrategy is consulted before sending a request in AsyncSinkWriter to decide if a request should be sent.
 */
public interface RateLimitingStrategy {
    /**
     * Tracks the information of requests being sent (e.g. to track the current inFlightMessages / requests)
     * @param requestInfo    Data class containing information on request being sent
     */
    void startedRequest(RequestInfo requestInfo);

	 /**
     * Tracks the information of requests completing (e.g. to track the current inFlightMessages / requests).
	 * Any dynamic scaling on failed requests should be done here too.
     * @param requestInfo    Data class containing information on request completed
     */
    void completedRequest(RequestInfo requestInfo);

 	 /**
     * Decides whether the next request should be sent.
     * @param requestInfo    Data class containing information on request being sent
     */
    boolean shouldBlock(RequestInfo requestInfo);

	 /**
     * Returns the nextmax batch size to be used by the AsyncSinkWriter.
     */
    int getNextBatchSizegetMaxBatchSize();
}

/**
 * This data class encapsulates the information of starting / completing requests.
 */
public final class RequestInfo {
	public int failedMessages;
	public int batchSize;
	public long requestStartTime;
}

...

To make it easier for sink implementers, we will also provide two out-of-the box implementations of CongestionControlRateLimitingStrategy, which uses the current AIMDStrategy to control in-flightMessagesflight messages. This mimics the existing behaviour of the AsyncSink.

...

We will also migrate the existing Firehose and Kinesis sinks to use the new interfaces proposed here. There will be no change in behaviour, since the CongestionControlStrategy CongestionControlRateLimitingStrategy configured will be the same as the current strategy.

...