THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
A new class with the following API will be introduced. Under the hood it, wraps, and delegates to the NumberSequenceSource
utilities.
...
Code Block | ||||
---|---|---|---|---|
| ||||
@Override public SourceReader<OUT, NumberSequenceSplit> createReader(SourceReaderContext readerContext) throws Exception { if (maxPerSecond > 0) { int parallelism = readerContext.getRuntimeContext().getNumberOfParallelSubtasks(); RateLimiter rateLimiter = new GuavaRateLimiter(maxPerSecond, parallelism); return new RateLimitedSourceReader<>( new MappingIteratorSourceReader<>(readerContext, generatorFunction), rateLimiter); } else { return new MappingIteratorSourceReader<>(readerContext, generatorFunction); } } |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
/** The interface that can be used to throttle execution of methods. */ |
...
public interface RateLimiter extends Serializable { |
...
/** |
...
* Acquire method is a blocking call that is intended to be used in places where it is required |
...
* to limit the rate at which results are produced or other functions are called. |
...
* |
...
* @return The number of milliseconds this call blocked its caller. |
...
* @throws InterruptedException The interrupted exception. |
...
*/ |
...
int acquire() throws InterruptedException; |
...
} |
This FLIP introduces a new DataGeneratorSource
class.
...