Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

/v1/sessions/:session_handle/operations/:operation_handle/result/:token

Verb: GET

Response code: 200 OK

Fetch a part of result for a flink job execution. If the result data is too large or the result is streaming, we can use this API to get a part of the result at a time. The initialized value of token is 0. The token in the next request must be the same as the token in the current request or must be equal to token (in the current request) + 1, otherwise the client will get an exception from the server. If multiple requests are executed with the same token, the result is the same. This design makes sure the client could get the result even if some errors occur in client. The client can get the next part of result using /v1/sessions/:session_id/jobs/:job_id/result/:{token+1} (which is the value of next_result_uri in the response data). If next_result_uri is empty, no more data is remaining in the server.

The server could drop the old data before current token. (The client successfully obtains those data)

We will introduce fetch_size or max_wait_time (to reach the fetch_size) for optimization in future.


The returned result has the same schema as the TableResult#getResolvedSchema.

Please refer to the Appendix about the transormation between the ResultSet and JSON.

Request body

{}

Response body

{

"result_type": "PAYLOAD",

"results": [ # currently, there is only one result now. If multiple queries is executed in a single job, there are many results.

{

"columns": [ # if the execution is successful

{

"name": "",

"type": {"type":  "BOOLEAN", "nullable": true}# string value of LogicalType

},

],

"data": [

["value", ], # a row data

]

},

],

"next_result_uri": /v1/sessions/:session_id/jobs/:job_id/result/:{token+1} # if not empty, uses this uri to fetch next part of result, else there is no more result.

"exception": {

      "root_cause": "....",

     "exception_stack": "..." 

}

}

...

We will collect more feedbacks to determine which features is more important to users.


Rejected Alternatives

TableInfo and UserDefinedFunctionInfo VS CatalogTable and CatalogFunction

...

Considering that not all LogicalType are serializable, our plan is much like how the LogicalTypeJsonSerializer does. Currently the LogicalType has 3 kinds:

  • Basic Type 

Data Type Name

JSON

CHAR
{"type": "CHAR", "nullable": true/false, "length": <LENGTH>}
VARCHAR
{"type": "VARCHAR", "nullable": true/false, "length": <LENGTH>}
STRING
{"type": "VARCHAR", "nullable": true/false, "length": <LENGTH>}
BOOLEAN
{"type": "BOOLEAN", "nullable": true/false} 
BINARY
{"type": "BINARY", "nullable": true/false, "length": <LENGTH>}
VARBINARY
{"type": "VARBINARY", "nullable": true/false, "length": <LENGTH>}
BYTES
{"type": "VARBINARY", "nullable": true/false, "length": <LENGTH>}
DECIMAL
{"type": "DECIMAL", "nullable": true/false, "precision": <LENGTH>, "scale": <SCALE>}
TINYINT
{"type": "TINYINT", "nullable": true/false}
SMALLINT
{"type": "SMALLINT", "nullable": true/false}
INTEGER
{"type": "INTEGER", "nullable": true/false}
BIGINT
{"type": "BIGINT", "nullable": true/false}
FLOAT
{"type": "FLOAT", "nullable": true/false}
DOUBLE
{"type": "DOUBLE", "nullable": true/false}
DATE
{"type": "DATE", "nullable": true/false}
TIME
{"type": "TIME", "nullable": true/false, "precision": <PRECISION>}
TIMESTAMP
{"type": "TIMESTAMP", "nullable": true/false, "precision": <PRECISION>}
TIMESTAMP_LTZ
{"type": "TIMESTAMP_LTZ", "nullable": true/false, "precision": <PRECISION>}
RAW
{"type": "RAW", "nullable": true/false, "class": <CLASS>, "specialSerializer": <SERIALIZER>} 

or

{"type": "RAW", "nullable": true/false, "class": <CLASS>, "externalDataType": ...}

...

Example

Code Block
languagetext
# example with
{
	"result_type": "PAYLOAD",
	"results": {
		"columns": [
			{
				"name": "id",
				"type": {"type":  "BIGINT", "nullable": false}
			},
   			{
				"name": "name",
				"type": {"type":  "VARCHAR", "nullable": true, "length": 300}
			},
		 	{
				"name": "birthday",
				"type": {"type":  "TIMESTAMP", "nullable": true, "precision": 3}
			}
		],

		"data": [
			[101, "Jay", "1990-01-12T12:00.12"], # a row data
			[102, "Jimmy", null]
		]
	},


	"next_result_uri": /v1/sessions/:session_id/jobs/:job_id/result/:{token+1} # if not empty, uses this uri to fetch next part of result, else there is no more result.

	"exception": {

      	"root_cause": "....",

     	"exception_stack": "..." 

	}

}



The design of the origin Gateway is in the 

...