Page History

...

Through the research summary and analysis, the overall implementation process is as follows:

Execution Flow

Image RemovedImage Added

Steps:

1. Create the sink table table in the catalog based on using the new API according to the schema of the query result, the table is visible but not persistent.
2. Start the Submit job and write the result to targetto the cluster.
3. After JM starts, create a table through Catalog.
4. If the job executes successfully, then make data visible.
5. If the job execution fails, then drop the sink table or delete data.(This capability requires runtime module support, such as hook, and SQL passes relevant parameters to the runtime module.)

Internal JM Hook Definition

Supported Job Mode

Support both streaming and batch mode.

...