Status
Current state: [Under Discussion]
Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
JIRA: Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14137 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14143 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-13510
Released: <Flink Version>
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
According to the docs, there may exist more than one attempt in a subtask, but there is no way to get the attempt history list in the REST API, users have no way to know if the subtask has failed before.
In the timeline page, the Web UI can only get and visualize the latest execution attempt of a subtask timeline, there is no way to get a failed attempt timeline in the current REST API.
Proposed Changes
Add the attempt history
- We can add the attempt history under the subtasks drawer on the job vertex page.
- Display all attempt timeline.
Frontend Design
Add view attempt history menu in the subtask, when popup an attempts history modal when user clicking it.
...
- get prior execution attempt
AccessExecution execution = executionVertex.getCurrentExecutionAttempt(); int currentAttemptNum = execution.getAttemptNumber(); JobID jobID = request.getPathParameter(JobIDPathParameter.class); JobVertexID jobVertexID = request.getPathParameter(JobVertexIdPathParameter.class); List<SubtaskExecutionAttemptDetailsInfo> allAttempts = new ArrayList<>(); allAttempts.add(SubtaskExecutionAttemptDetailsInfo.create(execution, metricFetcher, jobID, jobVertexID)); if (currentAttemptNum > 0) { for (int i = currentAttemptNum - 1; i >= 0; i--) { AccessExecution currentExecution = executionVertex.getPriorExecutionAttempt(i); if (currentExecution != null) { allAttempts.add(SubtaskExecutionAttemptDetailsInfo.create(currentExecution, metricFetcher, jobID, jobVertexID)); } } } |
- add SubtaskAllExecutionAttemptsDetailsHandler for failed attempt
- url /jobs/:jobid/vertices/:vertexid/subtasks/:subtaskIndex/attempts
- response:
{ "attempts" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo", "properties" : { "subtask" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } } } } } |
- add Collection<SubtaskTimeInfo> in SubtasksTimesHandler result, which comes from ExecutionVertex’s prior executions.
- url: /jobs/:jobid/vertices/:vertexid/subtasktimes
- response
{ "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo", "properties" : { "id" : { "type" : "string" }, "name" : { "type" : "string" }, "now" : { "type" : "integer" }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo:SubtaskTimeInfo", "properties" : { "subtask" : { "type" : "integer" }, "host" : { "type" : "string" }, "duration" : { "type" : "integer" }, "timestamps" : { "type" : "object", "additionalProperties" : { "type" : "integer" } }, "attempt-num": { "type" : "integer" }, "attempts-time-info": { "type": "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtaskAttemptTimeInfo", "properties" : { "subtask" : { "type" : "integer" }, "host" : { "type" : "string" }, "duration" : { "type" : "integer" }, "timestamps" : { "type" : "object", "additionalProperties" : { "type" : "integer" } }, "attempt-num": { "type" : "integer" }, } } } } } } } } |