Status
Current state: [Under Discussion]
...
Page properties | ||||
---|---|---|---|---|
|
...
...
|
...
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
When users click the vertex timeline, display all the subtask attempts timeline with subtaskId-host-attemptId.
REST API Design
- ArchivedExecutionVertex adds method to return prior executions.
- get prior execution attempt
...
AccessExecution execution = executionVertex.getCurrentExecutionAttempt();
int currentAttemptNum = execution.getAttemptNumber();
JobID jobID = request.getPathParameter(JobIDPathParameter.class);
JobVertexID jobVertexID = request.getPathParameter(JobVertexIdPathParameter.class);
List<SubtaskExecutionAttemptDetailsInfo> allAttempts = new ArrayList<>();
allAttempts.add(SubtaskExecutionAttemptDetailsInfo.create(execution, metricFetcher, jobID, jobVertexID));
if (currentAttemptNum > 0) {
for (int i = currentAttemptNum - 1; i >= 0; i--) {
AccessExecution currentExecution = executionVertex.getPriorExecutionAttempt(i);
if (currentExecution != null) {
allAttempts.add(SubtaskExecutionAttemptDetailsInfo.create(currentExecution, metricFetcher, jobID, jobVertexID));
}
}
}
- by ArchivedExecutionVertex.getPriorExecutionAttempts()
- add add SubtaskAllExecutionAttemptsDetailsHandler for failed attempt
- url /jobs/:jobid/vertices/:vertexid/subtasks/:subtaskIndex/attempts
- response:
{ "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskAllExecutionAttemptsDetailsInfo", "properties" : { "attempts" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo", "properties" : { "subtask" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } } } } } |
- add Collection<SubtaskTimeInfo> in SubtasksTimesHandler result, which comes from ExecutionVertex’s prior executions.
- add query parameter show-history, default value is false. if show-history is true return the information of attempt, else return emtpy array.
- url: /jobs/:jobid/vertices/:vertexid/subtasktimes
- response
"read-bytes" : {
{ "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo", "properties" : { "id" : { "type" : "stringinteger" }, "name" : { "typeread-bytes-complete" : "string"{ }, "now" : { "type" : "integerboolean" }, "subtasks" : { "typewrite-bytes" : "array",{ "items" : { "type" : "objectinteger", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo:SubtaskTimeInfo", }, "properties" : { "subtaskwrite-bytes-complete" : { "type" : "integerboolean" }, "hostread-records" : { "type" : "stringinteger" }, "durationread-records-complete" : { "type" : "integerboolean" }, "timestampswrite-records" : { "type" : "object"integer" }, "additionalPropertieswrite-records-complete" : { "type" : "integerboolean" } } }, "attempttaskmanager-numid" : { |
- In the 'subtasks' array we have objects of type SubtaskTimeInfo with only one added field 'attempt'.
- add query parameter show-history, default value is false. If show-history is true, information for all attempts including
previous ones will be returned - url: /jobs/:jobid/vertices/:vertexid/subtasktimes?show-history=true
- response:
{ "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtaskAttemptTimeInfoSubtasksTimesInfo", "properties" : { "id" : { "type" : "string" }, "propertiesname" : { "type" : "string" }, "hostnow" : { "type" : "integer" }, "subtasks" : { "type" : "stringarray", "items" : { }"type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo:SubtaskTimeInfo", "durationproperties" : { "subtask" : { "type" : "integer" }, "timestampshost" : { "type" : "objectstring", }, "additionalPropertiesduration" : { "type" : "integer" }, } "timestamps" : { }"type" : "object", "attempt-num"additionalProperties" : { "type" : "integer" } }, "attempt": { } "type" : "integer" } } } } } } } |
...