Versions Compared

Old Version 5

changes.mady.by.user Yijie Shen

Saved on

compared with

New Version 6

changes.mady.by.user Jianyun Zhao

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Pulsar as a streaming source with exactly-once guarantee.
  • Sink streaming results to Pulsar with atwith exactly-least-once semantics. 
  • Build upon Flink new Table API Type systemNew TableSource and TableSink interfaces (FLIP-3795), and can automatically (de)serialize messages with the help of Pulsar schemametadata (FLIP-107).
  • Integrate with Flink new Catalog API (FLIP-30), which enables the use of Pulsar topics as tables in Table API as well as SQL client.
  • Integrate with Flink new Source API (FLIP-27).
  • Integrate with Flink new Sink API (FLIP-143).
  • Integrate with Flink upsert-pulsar (FLIP-149).

Overall Design

Source: The exactly-once source should implement CheckpointedFunction to persist reading status (MessageId in Pulsar which uniquely identifies a message in topic) to state store, and could notify Pulsar of messages finished consumption by implementing CheckpointListener.

...