THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block | ||||
---|---|---|---|---|
| ||||
/** * This class represents a kind of partitioned data stream. For this stream, Each key group is a * partition, and the partition to which the data belongs is determined. */ public interface KeyedPartitionStream<K, T> { /** * Apply an operation to this {@link KeyedPartitionStream}. * * <p>This method is used to avoid shuffle after applying the process function. It is required * that for the same record, the new {@link KeySelector} must extract the same key as the * original {@link KeySelector} on this {@link KeyedPartitionStream}. * * @param processFunction to perform operation. * @param newKeySelector to select the key after process. * @return new {@link KeyedPartitionStream} with this operation. */ <OUT> KeyedPartitionStream<K, OUT> process( OneInputStreamProcessFunction<T, OUT> processFunction, KeySelector<OUT, K> newKeySelector); /** * Apply an operation to this {@link KeyedPartitionStream}; * * @param processFunction to perform operation. * @return new {@link NonKeyedPartitionStream} with this operation. */ <OUT> NonKeyedPartitionStream<OUT> process( OneInputStreamProcessFunction<T, OUT> processFunction); /** * Apply a two output operation to this {@link KeyedPartitionStream}. * * <p>This method is used to avoid shuffle after applying the process function. It is required * that for the same record, these new two {@link KeySelector}s must extract the same key as the * original {@link KeySelector}s on this {@link KeyedPartitionStream}. * * @param processFunction to perform two output operation. * @param keySelector1 to select the key of first output. * @param keySelector2 to select the key of second output. * @return new {@link TwoKeyedPartitionStreamTwoKeyedPartitionStreams} with this operation. */ <OUT1, OUT2> TwoKeyedPartitionStreams<K, OUT1, OUT2> process( TwoOutputStreamProcessFunction<T, OUT1, OUT2> processFunction, KeySelector<OUT1, K> keySelector1, KeySelector<OUT2, K> keySelector2); /** * Apply a two output operation to this {@link KeyedPartitionStream}. * * @param processFunction to perform two output operation. * @return new {@link TwoNonKeyedPartitionStreamTwoNonKeyedPartitionStreams} with this operation. */ <OUT1, OUT2> TwoNonKeyedPartitionStreams<OUT1, OUT2> process( TwoOutputStreamProcessFunction<T, OUT1, OUT2> processFunction); /** * Apply a two input operation to this and other {@link KeyedPartitionStream}. The two keyed * streams must have the same partitions, otherwise it makes no sense to connect them. * * @param other {@link KeyedPartitionStream} to perform operation with two input. * @param processFunction to perform operation. * @return new {@link NonKeyedPartitionStream} with this operation. */ <T_OTHER, OUT> NonKeyedPartitionStream<OUT> connectAndProcess( KeyedPartitionStream<K, T_OTHER> other, TwoInputNonBroadcastStreamProcessFunction<T, T_OTHER, OUT> processFunction); /** * Apply a two input operation to this and other {@link KeyedPartitionStream}.The two keyed * streams must have the same partitions, otherwise it makes no sense to connect them. * * <p>This method is used to avoid shuffle after applying the process function. It is required * that for the same record, the new {@link KeySelector} must extract the same key as the * original {@link KeySelector}s on these two {@link KeyedPartitionStream}s. * * @param other {@link KeyedPartitionStream} to perform operation with two input. * @param processFunction to perform operation. * @param newKeySelector to select the key after process. * @return new {@link KeyedPartitionStream} with this operation. */ <T_OTHER, OUT> KeyedPartitionStream<K, OUT> connectAndProcess( KeyedPartitionStream<K, T_OTHER> other, TwoInputNonBroadcastStreamProcessFunction<T, T_OTHER, OUT> processFunction, KeySelector<OUT, K> newKeySelector); /** * Apply a two input operation to this and other {@link BroadcastStream}. * * @param processFunction to perform operation. * @return new stream with this operation. */ <T_OTHER, OUT> NonKeyedPartitionStream<OUT> connectAndProcess( BroadcastStream<T_OTHER> other, TwoInputBroadcastStreamProcessFunction<T, T_OTHER, OUT> processFunction); /** * Apply a two input operation to this and other {@link BroadcastStream}. * * <p>This method is used to avoid shuffle after applying the process function. It is required * that for the record from non-broadcast input, the new {@link KeySelector} must extract the * same key as the original {@link KeySelector}s on the {@link KeyedPartitionStream}. For the * record from broadcast input, the output key from keyed partition itself instead of new key * selector, so it is safe already. * * @param other {@link BroadcastStream} to perform operation with two input. * @param processFunction to perform operation. * @param newKeySelector to select the key after process. * @return new {@link KeyedPartitionStream} with this operation. */ <T_OTHER, OUT> KeyedPartitionStream<K, OUT> connectAndProcess( BroadcastStream<T_OTHER> other, TwoInputBroadcastStreamProcessFunction<T, T_OTHER, OUT> processFunction, KeySelector<OUT, K> newKeySelector); /** * Coalesce this stream to a {@link GlobalStream}. * * @return the coalesced global stream. */ GlobalStream<T> global(); /** * Transform this stream to a new {@link KeyedPartitionStream}. * * @param keySelector to decide how to map data to partition. * @return the transformed stream partitioned by key. */ <NEW_KEY> KeyedPartitionStream<NEW_KEY, T> keyBy(KeySelector<T, NEW_KEY> keySelector); /** * Transform this stream to a new {@link NonKeyedPartitionStream}, data will be shuffled between * these two streams. * * @return the transformed stream after shuffle. */ NonKeyedPartitionStream<T> shuffle(); /** * Transform this stream to a new {@link BroadcastStream}. * * @return the transformed {@link BroadcastStream}. */ BroadcastStream<T> broadcast(); /** * Sink data from this stream. * * @param sink to receive data from this stream. */ void toSink(Sink<T> sink); /** * This class represents a combination of two {@link KeyedPartitionStream}. It will be used as * the return value of operation with two output. */ interface TwoKeyedPartitionStreams<K, T1, T2> { /** Get the first stream. */ KeyedPartitionStream<K, T1> getFirst(); /** Get the second stream. */ KeyedPartitionStream<K, T2> getSecond(); } } |
...
Similarly to source, we only supports sinkV2 based sink.
Move Related Classes to flink-core-api
The FLIP needs to move the following classes from flink-core into flink-core-api module:
Class Full Path |
---|
org.apache.flink.api.common.functions.Function |
org.apache.flink.api.java.functions.KeySelector |
Compatibility, Deprecation, and Migration Plan
...