Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


This page is meant as a template for writing a FLIP. To create a FLIP choose Tools->Copy on this page and modify with your content and replace the heading with the next FLIP number and a description of your issue. Replace anything in italics with your own description.

Page properties

Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".

Discussion threadhttps://lists.apache.org/thread/sls1196mmk25w8nm2qf585254nbjr9hd
Vote threadhere (<- link to https://lists.apache.org/list.html?dev@flink.apache.org)
JIRAhere (<- link to https://issues.apache.org/jira/browse/FLINK-XXXX)
/thread/rx435kvxctn6z6nf165vrg5c19nlpwcf
JIRA


Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyFLINK-33146
Release1.19Release<Flink Version>


Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

  1. Only hostname is provided to represent TaskManager location in some places (e.g. SubtaskCurrentAttemptDetailsHandler). However, in a containerized era, it is common to have multiple TMs on the same host, and port info is crucial to distinguish different TMs.
  2. Inconsistent naming of the field to represent TaskManager location: "host" is used in most places but "location" is also used in JobExceptions-related places.
  3. Inconsistent semantics of the "host" field: The semantics of the host field are inconsistent, sometimes it denotes hostname only while in other times it denotes hostname + port (which is also inconsistent with the name of "host"). To provide some examples:
PlacesSemantics of the "host" field
SubtaskCurrentAttemptDetailsHandlerHostname only
JobVertexTaskManagersHandlerHostname + port
SubtasksTimesHandlerHostname only

As a result, users find it difficult to identify TM-related problems and confusing to deal with the inconsistent naming and semantics of the fields in REST API. Similar issues have been raised independently multiple times, trying to address it:

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

Any handlers/interfaces in the entire REST API and Web UI that contains TaskManager location info.

  • SubtasksAllAccumulatorsHandler
  • SubtasksTimesHandler
  • SubtaskCurrentAttemptDetailsHandler (corresponds to the subtasks details panel of a job vertex on UI)
  • JobVertexTaskManagersHandler (corresponds to theTaskManger details panel of a job vertex on UI)

  • JobExceptionsHandler

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

  1. Use a field named "location" (already used in JobExceptionsInfoWithHistory) that represents TaskManager location Add a new string formatter method getEndpoint() to TaskManagerLocation and ArchivedTaskManagerLocation that prints in the form of "${hostname}:${port}" in a consistent mannerto align the string formatter used by REST API.
  2. Use a field named "endpoint" that represents TaskManager endpoint (host + port) using the newly added string formatter method.
  3. Update the front-end to adopt the new location endpoint field.
  4. Rename the column name from "Host" to "LocationEndpoint" on the Web UI to reflect the change that both hostname and port are displayed.
  5. Keep the old "host" field untouched for compatibility. Notice that the info stored in the old "host" field is inconsistent, sometimes only host is stored, sometimes host + port is stored, we 'll leave them as they were with a few comments to keep being compatible.

Compatibility, Deprecation, and Migration Plan

...

The the old "host" fields in REST API is left untouched for compatibility, they will be marked as deprecated and can be removed in the next major version (2.0after being kept for at least 2 minot releases).

Test Plan

Describe in few sentences how the FLIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

UTs and simple human-involved tests on UI should cover it well.

Rejected Alternatives

...

Not applicable.