You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Status

Current state: [Under Discussion]

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

JIRA:   Unable to render Jira issues macro, execution error.

Released: <Flink Version>

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Aligning the overview page of JobManager with the Flink's new memory model (FLIP-116).

Proposed Changes

This includes making JobManager's memory-related metrics available in the UI. Additionally, the configuration parameters should be exposed similar to the TaskManager's overview (see FLIP-102: Add More Metrics to TaskManager).

JVM

Flink Compose0

user conf key1

configuration key2

metric max3

metric used3

Heap

Framework Heap

jobmanager.memory.heap.size

memoryConfiguration.frameworkHeap

Status.JVM.Memory.Heap.Max

Status.JVM.Memory.Heap.Used

Off-Heap

Framework Off-Heapjobmanager.memory.off-heap.sizememoryConfiguration.frameworkOffHeapStatus.JVM.Memory.NonHeap.MaxStatus.JVM.Memory.NonHeap.Used

JVM MetaSpace

jobmanager.memory.jvm-metaspace.size

memoryConfiguration.jvmMetaspace

Status.JVM.Memory.Metaspace.Max
Unable to render Jira issues macro, execution error.

Status.JVM.Memory.Metaspace.Used

Unable to render Jira issues macro, execution error.

JVM Overhead

jobmanager.memory.jvm-overhead.min

memoryConfiguration.jvmOverhead

Status.JVM.Memory.NonHeap.Max - Status.JVM.Memory.Metaspace.Max

Status.JVM.Memory.NonHeap.Used - Status.JVM.Memory.Metaspace.Used

jobmanager.memory.jvm-overhead.max

0 The partitioning how it is described in org.apache.flink.runtime.clusterframework.TaskExecutorProcessSpec.
1 These are the configuration parameters used in the Flink configuration.
2 These are the Json paths to address the properties in the HTTP REST API response. Additionally, memoryConfiguration.totalFlinkMemory  and totalProcessMemory are exposed through the REST API.
3 The metrics which are exposed through the metrics endpoint.

Frontend Design

Display metrics inside JVM, outside JVM and GC

REST API proposal:

Metrics

The API can be used to retrieve the metrics for the JobManager: http://localhost:8081/jobmanager/metrics?get=Status.JVM.Memory.Heap.Max,Status.JVM.Memory.Heap.Used,Status.JVM.Memory.NonHeap.Max,Status.JVM.Memory.NonHeap.Used,Status.JVM.Memory.Metaspace.Max,Status.JVM.Memory.Metaspace.Used

Memory Configuration


Test Plan

Covered by unit tests.

  • No labels