THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
A new class with the following API will be introduced. Under the hood it, wraps, and delegates to the NumberSequenceSource utilities.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
@Override
public SourceReader<OUT, NumberSequenceSplit> createReader(SourceReaderContext readerContext)
throws Exception {
if (maxPerSecond > 0) {
int parallelism = readerContext.getRuntimeContext().getNumberOfParallelSubtasks();
RateLimiter rateLimiter = new GuavaRateLimiter(maxPerSecond, parallelism);
return new RateLimitedSourceReader<>(
new MappingIteratorSourceReader<>(readerContext, generatorFunction),
rateLimiter);
} else {
return new MappingIteratorSourceReader<>(readerContext, generatorFunction);
}
} |
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
/** The interface that can be used to throttle execution of methods. */ |
...
public interface RateLimiter extends Serializable { |
...
/** |
...
* Acquire method is a blocking call that is intended to be used in places where it is required |
...
* to limit the rate at which results are produced or other functions are called. |
...
* |
...
* @return The number of milliseconds this call blocked its caller. |
...
* @throws InterruptedException The interrupted exception. |
...
*/ |
...
int acquire() throws InterruptedException; |
...
} |
This FLIP introduces a new DataGeneratorSource class.
...