...
| Code Block |
|---|
| language | java |
|---|
| title | JobInfor.java |
|---|
|
/** The {@link JobInfo} represents the meta information of the job. */
public interface JobInfo {
/**
* Get the name of current job.
*
* @return the name of current job
*/
String getJobName();
/** Get the {@link ExecutionMode} of current job. */
ExecutionMode getExecutionMode();
/** Execution mode of this current job. */
enum ExecutionMode {
STREAMING,
BATCH
}
// We will gradually add more related methods here.
} |
TaskInfo
| Code Block |
|---|
| language | java |
|---|
| title | JobInforTaskInfo.java |
|---|
|
/** The {@link TaskInfo} represents the meta information of the task. */
public interface TaskInfo {
/**
* Get the parallelism of current task.
*
* @return the parallelism of this process function.
*/
int getParallelism();
/**
* Get the max parallelism of current task.
*
* @return The max parallelism.
*/
int getMaxParallelism();
/**
* Get the name of current task.
*
* @return The name of current task.
*/
String getTaskName();
// We will gradually add more related methods here.
} |
ProcessingTimeManager
| Code Block |
|---|
| language | java |
|---|
| title | JobInforProcessingTimeManager.java |
|---|
|
/**
* This is responsibility for managing runtime information related to processing time of process
* function.
*/
public interface ProcessingTimeManager {
/**
* Register a processing timer for this process function. onProcessingTimer method of this
* function will be invoked as callback if the timer expires.
*
* @param timestamp to trigger timer callback.
*/
void registerProcessingTimer(long timestamp);
/**
* Get the current processing time.
*
* @return current processing time.
*/
long currentProcessingTime();
} |
...
It should be noted that we can only register processing timer here. This is because we no longer unify event time and processing time into a single time service. Simply put, event Time can be implemented using the generalized watermark mechanism, and more detailed reasons have been explained in the Umbrella FLIP.
StateManager
| Code Block |
|---|
| language | java |
|---|
| title | JobInforStateManager.java |
|---|
|
/** This is responsibility for managing runtime information related to state of process function. */
public interface StateManager {
/**
* Get the key of current record.
*
* @return The key of current processed record for {@link KeyedPartitionStream}. {@link
* Optional#empty()} for other non-keyed stream.
*/
<K> Optional<K> getCurrentKey();
// The logic of getting states is omitted in this FLIP.
} |
The logic of getting states is omitted, but we will introduce it in detail in other sub-FLIPs later.
...
| Code Block |
|---|
| language | java |
|---|
| title | JobInforWatermarkManager.java |
|---|
|
/**
* This is responsibility for managing runtime information related to watermark of process function.
*/
public interface WatermarkManager {
// The logic of processing watermark is omitted in this FLIP.
} |
The logic of processing watermark is omitted, but we will introduce it in detail in other sub-FLIPs later.
...
| Code Block |
|---|
| language | java |
|---|
| title | JobInforRuntimeContext.java |
|---|
|
...
/**
* A RuntimeContext contains information about the context in which process functions are executed.
* Each parallel instance of the function will have a context through which it can access contextual
* information, such as the current key and execution mode. Through this context, we can also
* interact with the execution layer, such as getting state, emitting watermark, registering timer, etc.
*/
public interface RuntimeContext {
/** Get the {@link JobInfo} of this process function. */
JobInfo getJobInfo();
/** Get the {@link TaskInfo} of this process function. */
TaskInfo getTaskInfo();
/** Get the {@link ProcessingTimeManager} of this process function. */
ProcessingTimeManager getProcessingTimeManager();
/** Get the {@link StateManager} of this process function. */
StateManager getStateManager();
/** Get the {@link WatermarkManager} of this process function. */
WatermarkManager getWatermarkManager();
} |
ProcessFunction
At the same time, we should expand the implementation of Process Function. In more detail, we will introduce a new method called onProcessingTimer to all type of process functions. It is the callback for the timer registered through RuntimeContext#ProcessingTimeManager.
Note: The logic for processing watermark is omitted, but we will elaborate on it later in the future sub-FLIPs.
...