Versions Compared

Old Version 18

changes.mady.by.user Hongshun Wang

Saved on

compared with

New Version 19

changes.mady.by.user Hongshun Wang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Though the SingleThreadFetcherManager is annotated as Internal, it actually acts as some-degree public API, which is widely used in many connector projects:flink-connector-kafka,  flink-connector-mongodb, flink-cdc-connectors and soon. In flink-connector-kafka, KafkaSourceFetcherManager even extends SingleThreadFetcherManager.

  2. More over, even the constructor of SingleThreadMultiplexSourceReaderBase(which is PublicEvolving) includes the params of SingleThreadFetcherManager   and and FutureCompletingBlockingQueue. That means that the SingleThreadFetcherManager and FutureCompletingBlockingQueue  have already been exposed to users for a long time and are widely used.As shown in FLINK-31324, FLINK-28853 used to change the default constructor of SingleThreadFetcherManager.However, it influenced a lot. Finally, the former constructor was added back and marked asDeprecated.

    Code Block
    public SingleThreadMultiplexSourceReaderBase(
    FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue,
    SingleThreadFetcherManager<E, SplitT> splitFetcherManager,
    RecordEmitter<E, T, SplitStateT> recordEmitter,
    Configuration config,
    SourceReaderContext context)
{ super(elementsQueue, splitFetcherManager, recordEmitter, config, context); }



  3. The original design of SplitFetcherManager and its subclasses was to make them public to the Source developers. The goal is to let us take care of the threading model, while the Source developers can just focus on the SplitReader implementation. Therefore, I think making SplitFetcherManater / SingleThreadFetcherManager public aligns with the original design. That is also why these classes are exposed in the constructor of SourceReaderBase.


...