THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Currently, we only support adding FLIP-27 based source. The stream returned from `fromSource` method is Non-KeyedPartitionStream by default. If there is a clear key selecting strategy, the keyBy partitioning can be followed later. The connector part will be explained in more detail in future FLIP.
...
ProcessFunction
Process function is used to describe the processing logic of data. It is the key part for users to implement their job. Overall, we have a base interface for all user defined process functions that contains some life cycle methods, such as open and close. In addition, it also contains some common methods related to state and watermark, but we omit these methods here for simplicity, and we will introduce it in the corresponding sub-FLIPs.
...
OneInputStreamProcessFunction
| Code Block | ||||
|---|---|---|---|---|
| ||||
/** This class contains all logical related to process records from single input. */
public interface OneInputStreamProcessFunction<IN, OUT> extends ProcessFunction {
/**
* Process record and emit data through {@link Collector}.
*
* @param record to process.
* @param output to emit processed records.
* @param ctx, runtime context in which this function is executed.
*/
void processRecord(IN record, Collector<OUT> output, RuntimeContext ctx) throws Exception;
/**
* This is a life-cycle method indicates that this function will no longer receive any input
* data. This allowing the ProcessFunction to emit results at once rather than upon each record.
*
* @param output to emit record.
* @param ctx, runtime context in which this function is executed.
*/
default void endInput(Collector<OUT> output, RuntimeContext ctx) {}
} |
TwoInputStreamProcessFunction
| Code Block | ||||
|---|---|---|---|---|
| ||||
/** This class contains all logical related to process records from two input. */
public interface TwoInputStreamProcessFunction<IN1, IN2, OUT> extends ProcessFunction {
/**
* Process record from first input and emit data through {@link Collector}.
*
* @param record to process.
* @param output to emit processed records.
* @param ctx, runtime context in which this function is executed.
*/
void processRecordFromFirstInput(IN1 record, Collector<OUT> output, RuntimeContext ctx)
throws Exception;
/**
* Process record from second input and emit data through {@link Collector}.
*
* @param record to process.
* @param output to emit processed records.
* @param ctx, runtime context in which this function is executed.
*/
void processRecordFromSecondInput(IN2 record, Collector<OUT> output, RuntimeContext ctx)
throws Exception;
/**
* This is a life-cycle method indicates that this function will no longer receive any data from
* the first input. This allowing the ProcessFunction to emit results at once rather than upon
* each record.
*
* @param output to emit record.
* @param ctx, runtime context in which this function is executed.
*/
default void endFirstInput(Collector<OUT> output, RuntimeContext ctx) {}
/**
* This is a life-cycle method indicates that this function will no longer receive any data from
* the second input. This allowing the ProcessFunction to emit results at once rather than upon
* each record.
*
* @param output to emit record.
* @param ctx, runtime context in which this function is executed.
*/
default void endSecondInput(Collector<OUT> output, RuntimeContext ctx) {}
} |
TwoOutputStreamProcessFunction
| Code Block | ||||
|---|---|---|---|---|
| ||||
/** This class contains all logical related to process and emit records to two outputs. */
public interface TwoOutputStreamProcessFunction<IN, OUT1, OUT2> extends ProcessFunction {
/**
* Process and emit record to the first/second output through {@link Collector}s.
*
* @param record to process.
* @param output1 to emit processed records to the first output.
* @param output2 to emit processed records to the second output.
* @param ctx, runtime context in which this function is executed.
*/
void processRecord(
IN record, Collector<OUT1> output1, Collector<OUT2> output2, RuntimeContext ctx);
/**
* This is a life-cycle method indicates that this function will no longer receive any input
* data. This allowing the ProcessFunction to emit results at once rather than upon each record.
*
* @param output1 to emit record to the first output.
* @param output2 to emit record to the second output.
* @param ctx, runtime context in which this function is executed.
*/
default void endInput(
Collector<OUT1> output1, Collector<OUT2> output2, RuntimeContext ctx) {}
} |
RuntimeContext contains information about the context in which process functions are executed. It is currently just an empty interface but will be expanded later, such as supporting access to state, registering timers, etc. This part will be elaborated in the subsequent sub-FLIPs.
...