THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
/**
* This interface lets the sink developer to build a simple transactional sink topology pattern, which satisfies the HDFS/S3/Iceberg sink.
* This sink topology includes one {@link Writer} + one {@link Committer} + one {@link GlobalCommitter}.
* The {@link Writer} is responsible for producing the committable.
* The {@link Committer} is responsible for committing a single committable.
* The {@link GlobalCommitter} is responsible for committing an aggregated committable, which we called the global committable.
* And the parallelism of the {@link GlobalCommitter} is always 1.
* Both the {@link Committer} and the {@link GlobalCommitter} are optional.
* @param <InputT> The type of the sink's input
* @param <CommT> The type of the committable data
* @param <GlobalCommT> The type of the aggregated committable
* @param <WriterStateT> The type of the writer's state
*/
interface TransactionalSink<InputT, CommT, GlobalCommT, WriterStateT> {
/**
* Create a {@link Writer}.
* @param context the runtime context
* @param states the previous writers' state
* @return A sink writer
*/
Writer<InputT, CommT, WriterStateT> createWriter(InitContext context, List<WriterStateT> states);
/**
* @return a {@link Committer}
*/
Optional<Committer<CommT>> createCommitter();
/**
* @return a {@link GlobalCommitter}
*/
Optional<GlobalCommitter<CommT, GlobalCommT>> createGlobalCommitter();
Optional<SimpleVersionedSerializer<CommT>> getCommittableSerializer();
Optional<SimpleVersionedSerializer<GlobalCommT>> getGlobalCommittableSerializer();
Optional<SimpleVersionedSerializer<WriterStateT>> getWriterStateSerializer();
interface InitContext {
int getSubtaskId();
int getAttemptID();
MetricGroup metricGroup();
}
} |
...