THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
2. For FutureCompletingBlockingQueue, as a hindsight, it might be better to not expose it to the Source developers. They are unlikely to use it anywhere other than just constructing it. The reason that FutureCompletingBlockingQueue is currently exposed in the SourceReaderBase constructor is because both the SplitFetcherManager and SourceReaderBase need it. One way to hide the FutureCompletingBlockingQueue from the public API is to make SplitFetcherManager the only owner class of the queue, and expose some of its methods via SplitFetcherManager. This way, the SourceReaderBase can invoke the methods via SplitFetcherManager. I believe this also makes the code slightly cleaner.
In summary, this flip has 2 goals:
- Annotate SingleThreadFetcherManager as PublicEvolving.
- Shield FutureCompletingBlockingQueue from users and limit all operations on FutureCompletingBlockingQueue in FetcherManager.
Public Interfaces
Change SingleThreadFetcherManager, SplitFetcherManager, SplitFetcher and SplitFetcherTask from Internal to PublicEvolving .
...
| Code Block | ||
|---|---|---|
| ||
@PublicEvolving
public class SingleThreadFetcherManager<E, SplitT extends SourceSplit>
extends SplitFetcherManager<E, SplitT> {
@Deprecated
public SingleThreadFetcherManager(
FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue,
Supplier<SplitReader<E, SplitT>> splitReaderSupplier) {
this(elementsQueue, splitReaderSupplier, new Configuration());
}
@Deprecated
public SingleThreadFetcherManager(
FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue,
Supplier<SplitReader<E, SplitT>> splitReaderSupplier,
Configuration configuration) {
super(elementsQueue, splitReaderSupplier, configuration);
}
@Deprecated
@VisibleForTesting
public SingleThreadFetcherManager(
FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue,
Supplier<SplitReader<E, SplitT>> splitReaderSupplier,
Configuration configuration,
Consumer<Collection<String>> splitFinishedHook) {
super(elementsQueue, splitReaderSupplier, configuration, splitFinishedHook);
}
// todo: provide a new constructor without FutureCompletingBlockingQueue.
public SingleThreadFetcherManager(
Supplier<SplitReader<E, SplitT>> splitReaderSupplier) {
this(splitReaderSupplier, new Configuration());
}
public SingleThreadFetcherManager(
Supplier<SplitReader<E, SplitT>> splitReaderSupplier,
Configuration configuration) {
super(splitReaderSupplier, configuration);
}
public SingleThreadFetcherManager(
Supplier<SplitReader<E, SplitT>> splitReaderSupplier,
Configuration configuration,
Consumer<Collection<String>> splitFinishedHook) {
super(splitReaderSupplier, configuration, splitFinishedHook);
}
} |
...